Tensor Is the Might (zserge.com)

> A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object.

Erm... many would disagree. I think what he means is just a multidimensional array.

If one wants to add the capability to reason about shape and shape compatibility, Barry Jay's FiSh would be an interesting detour.

https://web.archive.org/web/20111015133833/http://www-staff....

This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops.

[Programming in FISh] https://link.springer.com/article/10.1007/s100090050037

[Towards Dynamic Shaping] https://www.researchgate.net/publication/265975794_Towards_D...

Why does does the diagram say tensors are 3D?

Cool, but I find rather than just shapes and indexes, tensors with labels are much easier to use and reason about. E.g.:

  {
      {user:bob, movie:"Heat"}:0.1,
      {user:alice, movie:"Frozen"}:0.9,
      {user:carol, movie:"Top Gun"}:0.3,
  }

https://docs.vespa.ai/en/ranking/tensor-user-guide.html

I just recently watched some (not all) of this video "coding a machine learning library in c from scratch" and seems like he's going through a similar process in this blog as this video. I would recommend watching the video to get an idea of what the fundamentals of a ML library look like. From someone who has recently been getting interested in actually writing ML code and trying to make sense of it myself (from the perspective of just a typical backend engineer) it was very interesting to see. Previously my experience with ML libs (PyTorch specific) was writing my own Mini-GPT and training it on a small dataset using my own GPU (5090). Cool to see the behind the scenes and took away some o the handwaveyness... https://www.youtube.com/watch?v=hL_n_GljC0I

> A tensor is nothing but a flat array of numbers

I'm so very, very tired of tech coopting rigorous mathematical terms.

I know there are different contexts, but a tensor is not a collection of numbers, in a mathematical sense. A vector is not a list of numbers. Such collections of numbers are representations of objects with very specific kinds of properties under coordinate transformations.

I think it genuinely damages people's ability to digest the mathematics to tell them first and foremost that these objects are collections of numbers.

> A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object.

Yikes! No.

I mean even for the intents and purposes of using this definition in ML, this might not be right.

I am trying not to be pedantic, so I will not go with the official/mathematical definition of a tensor as that could be incredibly confusing (look it up!!!).

But a tensor is a LOT more than that. Essentially it's a multilinear map that transforms a set of basis vectors in a certain way, and is coordinate agnostic.

This is not even half its definition so you can see how much the author left out.

Having said that, this is still a good way to start getting intuition into it and I urge the author to continue refining the definition as he/she learns more.

Disclaimer: MS in Math with concentration of GR.