Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful (for example, if you are averaging frequencies, you want the harmonic mean; if you are averaging interest rates, you want the geometric mean, etc.)
However, there are two big caveats:
(1) You can't use any symmetric binary function, for the mean to make sense, f must be at least injective and monotonic (and you probably want it to be continuous as well). If you pick a suitable f, many useful properties of the mean are respected.
(2) In general, for an arbitrary choice of f the resulting mean won't be homogenous, which is disqualifying for most practical applications (because, for example, the mean would not be guaranteed to have the same dimensions as the elements). Restricting ourselves to functions in the form $f(x) = x^a$ for $a \in \mathbb{R} \setminus 0$ yields a well-behaved collection of means (Holder means), guaranteed to be homogenous.
As a bonus, for any set $S$ of positive real numbers, if we let $GM(S)$ be the geometric mean of $S$ and $M_{a}(S)$ be the mean of $S$ for some $a$, you can prove that $\lim_{a \rightarrow 0} M_{a}(S) = GM(S)$.
Those caveats are just the beginning. You generally also want the mean to be associative, you want the function they call 'g' to be invertible, and the harmonic mean and geometric mean have the simpler and more useful interpretation of taking the average of the reciprocal and logarithm respectively.
All this article is doing, in a somewhat confused fashion, is trying to replace addition with another abelian (but not associative) operator. Which is fun, but ultimately not better than just assuming the arithmetic mean is going to be any help (which at least has some strong theorems that tell you it's going to converge to the expected value, usually).
> Great article, especially considering how careless most "data scientists" are when they assume the arithmetic mean is going to be meaningful
That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean.
Frequencies also don't add up.... But wavelength or time do. So their inverse can be added. So their inverses can have an arithmetic mean. So you can physically meaningfully do the arithmetic mean of their inverse, and inverse the result back again
> That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up.
If you're going down that route, which operations make sense and which don't boil down to the algebraic structure of the set of the possible values of the datatype. For example, you can't add dates but you can subtract dates, because dates have an affine structure, therefore the arithmetic mean of January 14 and October 13 is an operation that makes sense. Similarly with temperatures.
The thing is that it's sometimes tricky and unintuitive to figure out what you're allowed to do: ratios are a good example because you can add them, it just doesn't mean what most people expect it to mean.
I do agree that understanding your data goes a long way, but I believe making people more aware that tools other than "sum / n" exist is worthwhile, even though the article does have problems as I highlighted above.
Yes, but no temperatures cannot be substracted (nor added).
For dates I believe you're right because they are vectors (what you think is "an absolute date in the calendar" is in fact the time elapsed after 00h00 year=1 after christ, so it's in fact relative)
Both temperatures and dates have affine structure.
For dates it's really easy to see, because when you're talking about the calendar you're actually talking about two different types of objects, absolute time references (i.e. Thursday January 1, 1970 12:00 AM) and time durations (i.e. 45 minutes). (If you're familiar with Java, Instants and Durations). Time references cannot be added nor subtracted, but you can add a time duration to a time reference (midnight + 45 minutes = 12:45 AM of the same day) and the difference of two absolute time references is well-defined, unique and is a duration (1 AM - 12 AM = 1 hour). Furthermore, durations satisfy the axioms of a vector space (trivially, just pick a canonical unit, e.g. 1 second, and measure everything as a multiple/fraction of that).
The same holds with temperatures, with the unfortunate difference that we use the same notation for differences in temperatures (vectors of the affine space) and temperature references (base set of the affine space). For example, you can't add 18 celsius and 20 celsius, but saying that the temperature of a room increased by 2 celsius is meaningful, because "+2 celsius" is actually a vector. You can also add vectors to points, if the temperature increased by 2 more celsius, it would be 22 celsius. This does even have a physical meaning, and is independent of the frame of reference you pick for temperatures: a difference of 2 celsius is the same as a difference of 2 kelvin. (the same reasoning holds when replacing "celsius" with "fahrenheit" and "kelvin" with "rankine"). It is equally easy to verify temperature differences satisfy the axioms of a vector space, with the same argument as before.
That's your interpretation but it doesn't change the fact that physically you cannot add temperatures. You can take the numerical value of one temperature, add something to it, and re interpret it as a temperature; But the '+' operation on temperature does not have any physical meaning.
Replace "temperature" by "speed in kilometer/hour" in your sentence : "we cannot add 18 km/h and 20 km/h but I can add 2 km/h to 18 km/h because +2 km/h is a vector" it doesn't make sense to me
I don't follow your line of reasoning. You can add speeds (vector space), you can't add temperatures (points in an affine space cannot be added), but you can add temperatures and temperature differentials (points and vectors in an affine space).
Speeds are vectors in the first place, that's why you get to add them, while "temperature" denotes, depending on context, either a point (not addable) or a vector (addable).
This explains why you can't add temperatures, but the average temperature makes 100% sense and has a direct physical meaning.
Speed and temperature are intensive properties, they don't add up. You can remove the unit and do whatever you want mathematically and add back the physical unit, but it's not physics. Average temperature has no meaning it's a common fallacy, the only reason I ever got for that meaning is "my teacher told me it was OK !"
> Speed and temperature are intensive properties, they don't add up.
You can add speeds by their very definition. They are vectors in R^3 (or some other coordinate system, you get the point). The fact that addition of vectors is allowed is literally baked into the axioms of a vector space.
> Average temperature has no meaning it's a common fallacy
> You do realize math can handle units of measurements/dimensions just fine, do you?
Yes that's exactly what I'm talking about. Kelvin don't support the addition operator. You can pretend it's not Kelvin/Celsius, do something with the number, and add back the unit if you like, but it won't have any physical meaning.
... in an ideal gas model you can have a meaning adding and averaging k*T with k a magical constant that transforms your intensive property into extensive (energy)
> You can add speeds by their very definition
Definition of time derivation of the position function ? Yeah well [length]/[time] don't add up that's it. You need to first give a meaning to the addition operator of [length]/[time]. I'm talking in one dimension, of course if you can prove this then going 2D or 3D is easy
> That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean.
This isn't quite correct. You can take the arithmetic mean of interest rates, and it is meaningful. The ensemble average of interest rates might tell you the expectation of interest rates over a class of instruments at a point in time. However, if you want to know the compound growth rate of a single instrument or portfolio, then you're going to need to use the geometric mean.
You can average mathematically anything you want but it won't have the unit you want it attached, and the meaning you would like. Averages of interest rates, are not interest rates
In your specific example, the company is very interested in the ensemble average of their products growth rates, because that represents their rolled up revenue growth rate at a point in time (weighted by sales of each, of course).
An alternate example is the ensemble average of all corporate bonds in a particular ETF. People who hold that ETF are very interested in this number, because it represents the interest rate that they're earning when they hold that basket of bonds.
In both examples they are also interested in the geometric (time averaged) growth rate as well, because that represents their steady state earnings over time. What they really care about, in both examples, is the time-averaged rate of the ensemble rates.
So you are averaging amounts of money, so you are averaging things that can be added and you are not averaging interest rates. So you're not doing what you think you were doing and everything is good. (1.2+1.5)/2 doesn't have a meaning but (1.2moneyA+1.5moneyB)/2=moneyAB and growth rate has nothing to do here
> because it represents the interest rate that they're earning when they hold that basket of bonds
No it doesn't. Look at my example, you would think that the arithmetic average growth rate is 1.35. But now if I told you product A is in fact made up of two subproducts, one is not growing (1) and the others is growing a lot (2), you would compute a new average of (1+2+1.5)/3=1.5
> So you are averaging amounts of money, so you are averaging things that can be added and you are not averaging interest rates. So you're not doing what you think you were doing and everything is good. (1.2+1.5)/2 doesn't have a meaning but (1.2moneyA+1.5moneyB)/2=moneyAB and growth rate has nothing to do here
You're taking a weighted average of interest rates. You can call it an amount of money if you like, but you can also call it a weighted average of interest rates. This is no different from a weighted average of time-varying interest rates over irregular intervals of time.
They claim the inverse function of f(a,b) = a + b, is f(a,b) = a - b. Surely this is wrong. By my limited understaning of the basic structures of functions; when f: R^2 -> R, then f inverse ought to be R -> R^2 right? And when f is just addition there can’t be an inverse solving for both a and b.
(If one of a and b were fixed, it’d be way easier, that is, if f(a) = a + b, then g(a) = a - b is an inverse of f)
Yes, I noticed this from the statement "apply that binary operation to one number from the list". They're basically treating f as a function from R to R.
I don't quite follow the example either. For a list (2, 4, 6), the function is applied to the first number and the "inverse" function (though it's really not) on the last number of the list, with an argument of b=2. I must be missing something in the argument -- I'm wondering, why the choice of b=2? And was the function not applied to the number 4?
Edit:
I believe the author was trying to show (albeit incorrectly) the generalization of an arithmetic mean using f(x) = ax + b, with a = 1 and b = 2 (not f(a,b) as stated; plus the inverse function was applied incorrectly at the element level). The f-mean (Mf) is defined as:
Mf(x₁,...,xₙ) = f⁻¹( (1/n) ∑ₖⁿ f(xₖ) )
where the function f is injective and continuous. Different choices of f result in different means. This wikipedia article provides relevant details on the f-mean.
I was having a conversation on HN a few days ago about how the mean of a set can be seen as minimizer of an optimization problem -- roughly, it can be interpreted as that location that minimizes the 'distance' to all other points in a set. By choosing the 'distance' appropriately one can get all sorts of fun generalizations that includes all mentioned in the post. The familiar squared Euclidean distance yields the familiar arithmetic mean.
One fruitful family of such 'distances' that lead to arithmetic mean, geometric mean, Holder's mean [1] and beyond etc is the Bregman divergence [0]. The sort of means that fall out from it have the form
f⁻¹ ( 1/n ∑ᵢ₌₁ⁿ f(xᵢ) ) where f is a smooth monotonically increasing function. By f⁻¹ I mean the inverse function, not the reciprocal.
I really think the most "understandable" derivation of the arithmetic mean is that as a maximum-likelihood estimator of the mean of gaussian IID random variables. It literally requires a touch of calculus, and it pops out beautifully. Of course, proving it in general is a bit more involved, but...
One can do without calculus in this case. Using the fact that a squared expression is positive one can show that the arithmetic mean is indeed the maximum likelihood estimator. For the general case one cant avoid calculus.
31 comments
[ 4.2 ms ] story [ 76.5 ms ] threadHowever, there are two big caveats:
(1) You can't use any symmetric binary function, for the mean to make sense, f must be at least injective and monotonic (and you probably want it to be continuous as well). If you pick a suitable f, many useful properties of the mean are respected.
(2) In general, for an arbitrary choice of f the resulting mean won't be homogenous, which is disqualifying for most practical applications (because, for example, the mean would not be guaranteed to have the same dimensions as the elements). Restricting ourselves to functions in the form $f(x) = x^a$ for $a \in \mathbb{R} \setminus 0$ yields a well-behaved collection of means (Holder means), guaranteed to be homogenous. As a bonus, for any set $S$ of positive real numbers, if we let $GM(S)$ be the geometric mean of $S$ and $M_{a}(S)$ be the mean of $S$ for some $a$, you can prove that $\lim_{a \rightarrow 0} M_{a}(S) = GM(S)$.
All this article is doing, in a somewhat confused fashion, is trying to replace addition with another abelian (but not associative) operator. Which is fun, but ultimately not better than just assuming the arithmetic mean is going to be any help (which at least has some strong theorems that tell you it's going to converge to the expected value, usually).
I genuinely smiled when the article says "any function". "Any function" in math means any function :-)
There's a reason why theorems involving arbitrary functions start with a list of qualifiers as hypotheses.
That comes from the fact that they don't understand the data they are dealing with (as you said with dimensions). You cannot use the arithmetic mean on interest rates because it does not make sense to add them up. If you just go for the geometric mean "because it's a rate and I have read online I should use geometric mean" then you won't learn why you need to use the geometric mean.
Frequencies also don't add up.... But wavelength or time do. So their inverse can be added. So their inverses can have an arithmetic mean. So you can physically meaningfully do the arithmetic mean of their inverse, and inverse the result back again
If you're going down that route, which operations make sense and which don't boil down to the algebraic structure of the set of the possible values of the datatype. For example, you can't add dates but you can subtract dates, because dates have an affine structure, therefore the arithmetic mean of January 14 and October 13 is an operation that makes sense. Similarly with temperatures.
The thing is that it's sometimes tricky and unintuitive to figure out what you're allowed to do: ratios are a good example because you can add them, it just doesn't mean what most people expect it to mean.
I do agree that understanding your data goes a long way, but I believe making people more aware that tools other than "sum / n" exist is worthwhile, even though the article does have problems as I highlighted above.
For dates I believe you're right because they are vectors (what you think is "an absolute date in the calendar" is in fact the time elapsed after 00h00 year=1 after christ, so it's in fact relative)
For dates it's really easy to see, because when you're talking about the calendar you're actually talking about two different types of objects, absolute time references (i.e. Thursday January 1, 1970 12:00 AM) and time durations (i.e. 45 minutes). (If you're familiar with Java, Instants and Durations). Time references cannot be added nor subtracted, but you can add a time duration to a time reference (midnight + 45 minutes = 12:45 AM of the same day) and the difference of two absolute time references is well-defined, unique and is a duration (1 AM - 12 AM = 1 hour). Furthermore, durations satisfy the axioms of a vector space (trivially, just pick a canonical unit, e.g. 1 second, and measure everything as a multiple/fraction of that).
The same holds with temperatures, with the unfortunate difference that we use the same notation for differences in temperatures (vectors of the affine space) and temperature references (base set of the affine space). For example, you can't add 18 celsius and 20 celsius, but saying that the temperature of a room increased by 2 celsius is meaningful, because "+2 celsius" is actually a vector. You can also add vectors to points, if the temperature increased by 2 more celsius, it would be 22 celsius. This does even have a physical meaning, and is independent of the frame of reference you pick for temperatures: a difference of 2 celsius is the same as a difference of 2 kelvin. (the same reasoning holds when replacing "celsius" with "fahrenheit" and "kelvin" with "rankine"). It is equally easy to verify temperature differences satisfy the axioms of a vector space, with the same argument as before.
Replace "temperature" by "speed in kilometer/hour" in your sentence : "we cannot add 18 km/h and 20 km/h but I can add 2 km/h to 18 km/h because +2 km/h is a vector" it doesn't make sense to me
Speeds are vectors in the first place, that's why you get to add them, while "temperature" denotes, depending on context, either a point (not addable) or a vector (addable).
This explains why you can't add temperatures, but the average temperature makes 100% sense and has a direct physical meaning.
> Speed and temperature are intensive properties, they don't add up.
You can add speeds by their very definition. They are vectors in R^3 (or some other coordinate system, you get the point). The fact that addition of vectors is allowed is literally baked into the axioms of a vector space.
> Average temperature has no meaning it's a common fallacy
Yes, average temperature does have unambiguous meaning: https://en.wikipedia.org/wiki/Boltzmann_constant
> You can remove the unit and do whatever you want mathematically and add back the physical unit, but it's not physics.
You do realize math can handle units of measurements/dimensions just fine, do you?
Yes that's exactly what I'm talking about. Kelvin don't support the addition operator. You can pretend it's not Kelvin/Celsius, do something with the number, and add back the unit if you like, but it won't have any physical meaning.
> Yes, average temperature does have unambiguous meaning: https://en.wikipedia.org/wiki/Boltzmann_constant
... in an ideal gas model you can have a meaning adding and averaging k*T with k a magical constant that transforms your intensive property into extensive (energy)
> You can add speeds by their very definition
Definition of time derivation of the position function ? Yeah well [length]/[time] don't add up that's it. You need to first give a meaning to the addition operator of [length]/[time]. I'm talking in one dimension, of course if you can prove this then going 2D or 3D is easy
This isn't quite correct. You can take the arithmetic mean of interest rates, and it is meaningful. The ensemble average of interest rates might tell you the expectation of interest rates over a class of instruments at a point in time. However, if you want to know the compound growth rate of a single instrument or portfolio, then you're going to need to use the geometric mean.
If product A grows by 20% and product B grows by 50%, what are you going to do with 1.2 and 1.5 exactly ? (1.2+1.5)/2 has what meaning ?
An alternate example is the ensemble average of all corporate bonds in a particular ETF. People who hold that ETF are very interested in this number, because it represents the interest rate that they're earning when they hold that basket of bonds.
In both examples they are also interested in the geometric (time averaged) growth rate as well, because that represents their steady state earnings over time. What they really care about, in both examples, is the time-averaged rate of the ensemble rates.
So you are averaging amounts of money, so you are averaging things that can be added and you are not averaging interest rates. So you're not doing what you think you were doing and everything is good. (1.2+1.5)/2 doesn't have a meaning but (1.2moneyA+1.5moneyB)/2=moneyAB and growth rate has nothing to do here
> because it represents the interest rate that they're earning when they hold that basket of bonds
No it doesn't. Look at my example, you would think that the arithmetic average growth rate is 1.35. But now if I told you product A is in fact made up of two subproducts, one is not growing (1) and the others is growing a lot (2), you would compute a new average of (1+2+1.5)/3=1.5
You're taking a weighted average of interest rates. You can call it an amount of money if you like, but you can also call it a weighted average of interest rates. This is no different from a weighted average of time-varying interest rates over irregular intervals of time.
(If one of a and b were fixed, it’d be way easier, that is, if f(a) = a + b, then g(a) = a - b is an inverse of f)
Edit:
I believe the author was trying to show (albeit incorrectly) the generalization of an arithmetic mean using f(x) = ax + b, with a = 1 and b = 2 (not f(a,b) as stated; plus the inverse function was applied incorrectly at the element level). The f-mean (Mf) is defined as:
Mf(x₁,...,xₙ) = f⁻¹( (1/n) ∑ₖⁿ f(xₖ) )
where the function f is injective and continuous. Different choices of f result in different means. This wikipedia article provides relevant details on the f-mean.
https://en.wikipedia.org/wiki/Quasi-arithmetic_mean
One fruitful family of such 'distances' that lead to arithmetic mean, geometric mean, Holder's mean [1] and beyond etc is the Bregman divergence [0]. The sort of means that fall out from it have the form
f⁻¹ ( 1/n ∑ᵢ₌₁ⁿ f(xᵢ) ) where f is a smooth monotonically increasing function. By f⁻¹ I mean the inverse function, not the reciprocal.
[0] https://en.wikipedia.org/wiki/Bregman_divergence
[1] https://en.wikipedia.org/wiki/Generalized_mean
[2] https://news.ycombinator.com/item?id=25221587