I have been trying for a while to put irregular things (point clouds) in regular grids, to use the grid itself as a data structure. Not because it is better, but just because grids are more structured than e.g. trees and thus easier integrate into ML pipelines. It all started here, with simple multi-indexing:
Then I realised that to make the multi-indexing procedure more effective, I need some kind of preprocessing, to make the points flat (2d insteed of 3d). When they are already flat I can sort them better.
Then I heard about mesh flattening, https://geometrycollective.github.io/boundary-first-flatteni...
geometry images https://hhoppe.com/proj/gim/ and thought: this is something I need to try with point clouds, whenever they have a latent surfacic distribution. And well, it seems to work. At least now I can have bunnys dragon and elephants all to fit in the grid and not getting half of the legs, ears or whatever lost in the process.
It's realy the idea that Flat 3D object <-> Image, you can get a look at
what the "founders" of geometry images said about that,
and how they apply it to simplify some mesh pipelines (hope it will feed your active curiosity !):
https://hhoppe.com/proj/gim/
What I bring upon geometry images with my project, is the same idea for point clouds.
replace point[42] (index = 42) with point[row, col] (row, col of a learned matrix structure that store the points. Thus, the point cloud is stored as an image, and the pixels are the points).
Promising application is point cloud compression, but I can think of others: matrix storage is maybe not as accurate as e.g. kdtree storage, but very convenient to use, eg for neighbor querys, convolution...
10 comments
[ 9.2 ms ] story [ 403 ms ] threadhttps://news.ycombinator.com/item?id=49042861
Then I realised that to make the multi-indexing procedure more effective, I need some kind of preprocessing, to make the points flat (2d insteed of 3d). When they are already flat I can sort them better.
Then I heard about mesh flattening, https://geometrycollective.github.io/boundary-first-flatteni... geometry images https://hhoppe.com/proj/gim/ and thought: this is something I need to try with point clouds, whenever they have a latent surfacic distribution. And well, it seems to work. At least now I can have bunnys dragon and elephants all to fit in the grid and not getting half of the legs, ears or whatever lost in the process.
Opposite of snark: active curiosity :)
and how they apply it to simplify some mesh pipelines (hope it will feed your active curiosity !): https://hhoppe.com/proj/gim/
What I bring upon geometry images with my project, is the same idea for point clouds.
replace point[42] (index = 42) with point[row, col] (row, col of a learned matrix structure that store the points. Thus, the point cloud is stored as an image, and the pixels are the points).
Promising application is point cloud compression, but I can think of others: matrix storage is maybe not as accurate as e.g. kdtree storage, but very convenient to use, eg for neighbor querys, convolution...
import flatpoints as fp
import pyjpegxl
import os
#============ Raw point cloud =============================
xyz = fp.load("bunny") #Required format: numpy array, (N, 3)
#============ Fit the flattener ===========================
single_face = True #Or false -> two images (left + right)
flat = fp.Flat(xyz, single_face = single_face)
#============ Build one (single_face) resp 2 images: ======
images = flat.to_image(xyz) #(1, n, n, 3) or (2, n, n, 3)
left_image = images[0] #(n, n, 3), right_image = images[1]
#/!\, normalisation, needed for pyjpegxl:
b = xyz.min(axis = 0)
a = xyz.max(axis = 0) - b
left_image_normalised = (left_image - b[None, None])/a[None, None]
#============ Compression, Decompression ==================
pyjpegxl.imwrite(f"compressed.jxl", left_image_normalised, quality = 98)
_, image_decompressed = pyjpegxl.imread(f"compressed.jxl")
xyz_decompressed = a*image_decompressed.reshape(-1, 3) + b #basically, that's it.
#============ Visual inspection ============================
_ = fp.Flat(xyz_decompressed, single_face = single_face).plot()
#============ Memory saving ============================
memory_before = xyz.nbytes
memory_after = os.path.getsize('compressed.jxl')
print(f"Initial memory: {memory_before / 1024:.2f} KB")
print(f"New memory: {memory_after / 1024:.2f} KB")
- where is x, y and z pointed towards and what is its range