The jump from spherical harmonics to eigenfunctions on a general mesh, and the specific example mesh chosen, might be the finest mathematical joke I've seen this decade.
Isn't this the opposite way ? Vectors are functions whose input space are discrete dimensions. Let's not pretend going from natural numbers to real is "simple", reals numbers are a fascinating non-obvious math discovery. And also the passage from a few numbers to all natural numbers (aleph0) is non obvious. So basically we have two alephs passages to transforms N-D vectors as functions over reals.
A few questions occur to me while reading this, which I am far from qualified to answer:
- How much of this structure survives if you work on "fuzzy" real numbers? Can you make it work? Where I don't necessarily mean "fuzzy" in the specific technical sense, but in any sense in which a number is defined only up to a margin of error/length scale, which in my mind is similar to "finitism", or "automatic differentiation" in ML, or a "UV cutoff" in physics. I imagine the exact definition will determine how much vectorial structure survives. The obvious answer is that it works like a regular Fourier transform but with a low-pass filter applied, but I imagine this might not be the only answer.
- Then if this is possible, can you carry it across the analogy in the other direction? What would be the equivalent of "fuzzy vectors"?
- If it isn't possible, what similar construction on the fuzzy numbers would get you to the obvious endpoint of a "fourier analysis with a low pass filter pre-applied?"
- The argument arrives at fourier analysis by considering an orthonormal diagonalization of the Laplacian. In linear algebra, SVD applies more generally than diagonalizations—is there an "SVD" for functions?
The author asserts vectors are functions, specifically a function that takes an index and returns a value. He notes that as you increase the number of indices, a vector can contain an arbitary function (he focuses on continuous, real-valued functions).
It's fun to simulate one thing with another, but there is a deeper and more profound sense in which vectors are functions in Clifford Algebra, or Geometric Algebra. In that system, vectors (and bi-vectors...k-vectors) are themselves meaningful operators on other k-vectors. Even better, the entire system generalizes to n-dimensions, and decribes complex numbers, 2-d vectors, quaternions, and more, essentially for free. (Interestingly, the primary operation in GA is "reflection", the same operation you get in quantum computing with the Hadamard gate)
Thanks for mentioning this. I think I encountered Geometric Algebra in a video related to making video games. It's this one (Why you can't multiply vectors): https://www.youtube.com/watch?v=htYh-Tq7ZBI
Yeah, and it's such a boring thing to write about.
Given an vector space V with (+, ), you can define the vector space over functions F whose codomain is V and where F.+ and F. both take two functions as argument and return another function applying V.+ or V.* on the result. All the linear algebra properties come from the original vector space. Hence it is boring.
An engineering, signal processing extension/perspective:
An infinite sequence approximates a general function, as described in the article (see the slider bar example). In signal processing applications, functions can be considered (or forced) to be bandlimited so a much lower-order representation (i.e. vector) suffices:
- The subspace of bandlimited functions is much smaller than the full L^2 space
- It has a countable orthonormal basis (e.g., shifted sinc functions)
- The function can be written as (with sinc functions):
I will need to read through the rest of the article later, but the initial intuition building is a bit sloppy. None of those vectors drawn in the initial examples belong to the same vector space. Vectors need to emanate from the same origin to be considered as part of the same vector space.
The author seems to be a great educator and computer scientist, much respect to his work. But from what I can gather, although I'd love to study more infinite sized matrices, he proved / showed nothing in this article. What he wrote is not true at all, they are only analogies and not rigorous maths. Functions are not vectors. But finite polynomials are vectors yes, this is trivial.
Let's see: Let A be a non-empty set, N
the set of positive whole numbers, and X
the set of all functions f
f: A --> N
with usual notation.
Assume as is common, the scalars are the
set of real numbers, but the set of complex
numbers will also do.
So, is X a vector space and, thus, each
f in X a vector?
No, since -f is not in X. Neither is
(1/2)f.
Some references (with TeX markup):
Paul R.\ Halmos,
{\it Finite-Dimensional Vector Spaces, Second Edition\/}
linear algebra treated as functional analysis.
Walter Rudin,
{\it Real and Complex Analysis\/}
with Lebesgue integration and, then,
Banach and Hilbert vector spaces.
Walter Rudin,
{\it Functional Analysis\/}
with Fourier theory.
Jacques Neveu,
{\it Mathematical Foundations of the Calculus of Probability\/}
with random variables, that is,
functions from a probability space to,
usually, the set of real numbers with
convergence results, building on the
work A. Kolmogorov building on the work of
H. Lebesgue.
23 comments
[ 3.8 ms ] story [ 38.9 ms ] thread(This is where I learned at least half of the math on this page: theoretical chemistry.)
- How much of this structure survives if you work on "fuzzy" real numbers? Can you make it work? Where I don't necessarily mean "fuzzy" in the specific technical sense, but in any sense in which a number is defined only up to a margin of error/length scale, which in my mind is similar to "finitism", or "automatic differentiation" in ML, or a "UV cutoff" in physics. I imagine the exact definition will determine how much vectorial structure survives. The obvious answer is that it works like a regular Fourier transform but with a low-pass filter applied, but I imagine this might not be the only answer.
- Then if this is possible, can you carry it across the analogy in the other direction? What would be the equivalent of "fuzzy vectors"?
- If it isn't possible, what similar construction on the fuzzy numbers would get you to the obvious endpoint of a "fourier analysis with a low pass filter pre-applied?"
- The argument arrives at fourier analysis by considering an orthonormal diagonalization of the Laplacian. In linear algebra, SVD applies more generally than diagonalizations—is there an "SVD" for functions?
It's fun to simulate one thing with another, but there is a deeper and more profound sense in which vectors are functions in Clifford Algebra, or Geometric Algebra. In that system, vectors (and bi-vectors...k-vectors) are themselves meaningful operators on other k-vectors. Even better, the entire system generalizes to n-dimensions, and decribes complex numbers, 2-d vectors, quaternions, and more, essentially for free. (Interestingly, the primary operation in GA is "reflection", the same operation you get in quantum computing with the Hadamard gate)
Given an vector space V with (+, ), you can define the vector space over functions F whose codomain is V and where F.+ and F. both take two functions as argument and return another function applying V.+ or V.* on the result. All the linear algebra properties come from the original vector space. Hence it is boring.
Functions on a countable domain are sequences.
An infinite sequence approximates a general function, as described in the article (see the slider bar example). In signal processing applications, functions can be considered (or forced) to be bandlimited so a much lower-order representation (i.e. vector) suffices:
- The subspace of bandlimited functions is much smaller than the full L^2 space - It has a countable orthonormal basis (e.g., shifted sinc functions) - The function can be written as (with sinc functions):
x(t) = \sum_{n=-\infty}^{\infty} f(nT) \cdot \text{sinc}\left( \frac{t - nT}{T} \right)
- This is analogous to expressing a vector in a finite-dimensional subspace using a basis (e.g. sinc)
Discrete-time signal processing is useful for comp-sci applications like audio, SDR, trading data, etc.
Polynomials come to mind.
Let's see: Let A be a non-empty set, N the set of positive whole numbers, and X the set of all functions f
f: A --> N
with usual notation.
Assume as is common, the scalars are the set of real numbers, but the set of complex numbers will also do.
So, is X a vector space and, thus, each f in X a vector?
No, since -f is not in X. Neither is (1/2)f.
Some references (with TeX markup):
Paul R.\ Halmos, {\it Finite-Dimensional Vector Spaces, Second Edition\/}
linear algebra treated as functional analysis.
Walter Rudin, {\it Real and Complex Analysis\/}
with Lebesgue integration and, then, Banach and Hilbert vector spaces.
Walter Rudin, {\it Functional Analysis\/}
with Fourier theory.
Jacques Neveu, {\it Mathematical Foundations of the Calculus of Probability\/}
with random variables, that is, functions from a probability space to, usually, the set of real numbers with convergence results, building on the work A. Kolmogorov building on the work of H. Lebesgue.