Transformations like these have wide applicability. Performing optics in software so to speak, is more flexible and cheaper than grinding lenses and building special cameras.
I was charged once, with transforming images for projection onto a sphere through special lenses. I didn't know the lens transform, the manufacturer kept that secret. I did know that the intensity across the spherical surface was of uniform brightness - the manufacturer boasted about this. With that bound I could guess the transform, as each pixel of the source image would have to illuminate an equal surface area of sphere once projected.
The manufacturer advertised a service to do this transform on your image, but they wanted $10K for each run! Once I presented my insights and a little math, they capitulated and did it at cost. I never got to write my projection software in the end.
Could you give me some pointers on where I could start studying about the math involved? Are those topics usually covered by good books on computer graphics?
I loved "Multiple View Geometry" by Hartley & Zisserman, but also "Computer Vision" by Rick Szelisky is pretty comprehensible and covers a lot of (pre-ML) stuff.
Sounds like a classic f-theta lens design. I'm sure you were right about your transform, but it's also possible to equalize illumination by other means, like apodizing filters.
> If "straight" lines are not straight that normally means the fisheye center or radius are not specified correctly or the angle is not defined correctly.
Is it possible to make the computer guess the correct value with AI? I guess a method is to identify the artificial objects like walls and doors, and assume they have straight lines. The nice curved tops of the windows may be a problem. And probably there must be a special case for circles like the clock.
For a single image, you'd need some heuristics to distinguish curved objects from camera distortions. If you have multiple frames, you can SLAM your way out of the problem (essentially, match features and solve a big fit which optimized feature positions, camera positions and camera distortions). But by far the easiest option is to just calibrate it, through https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.h... or one of the many alternative methods.
Calibrating using known objects should be possible. Most road sign dimensions are specified in national laws, motor vehicle headlights must be a specific distance apart, etc.
The applications of this are interesting to say the least.
Something i am currently working on is using fisheye images of the sky to predict cloud motion and to then estimate solar PV power output in the short term i.e. one to five minutes out.
It sounds simple but it isn’t yet even a child can observe cloud motion and predict to some degree when the disc of the sun will be obscured. Part of the inaccuracy is in the distortion caused by the fisheye lens.
I would be interested in seeing how well this works for subjects at infinity i.e. a fisheye lens pointed at the sky.
Paul Bourke's webpage is a true internet gem for 3D reference information. I don't know how many times I must've landed on his page on 3D file format specifications.
A few years ago, I was working for a startup in Pakistan which was making small (5-10 person) portable planetariums. The idea was to make astronomy education accessible to people from every socio-economic background. Paul's work on this [1] was amazingly helpful to get our projection done well, and put on successful shows.
Also, shout out to the European Space Agency for releasing high quality Planetarium documentaries under the CC license [2].
Back around 2006 I used "panorama tools" to do the same sort of thing. It had this cool feature of being able to work out lens calibration if it had two photos from the same viewpoint but different angles. It did however require careful tagging of reference points on each image.
The logical successor to panorama tools is hugin these days, and yes it is able to perform these transformations very easily. The control point generation can be done automatically now, although I admit it might struggle a little with fisheye images.
Paul was kind enough to meet with me a few times, probably over a decade ago now, when I was a lost student trying to work out what ideas were worth exploring (I guess I still am).
He showed me some photogramatery work he was working on, I was amazed at how smooth everything was running given the precision and having worked with highly detailed tins and pointclouds myself on that days high end hardware. I asked him what kernel/engine he was using. He casually said it was his own. Which (having done toy ray tracing in C and dabling in openGL) blew my mind but also made me realise it would be absolutely necessary in order to do this kind of work with the type of control he needed. Everything from the ground up.
One of the few academics/researchers I've met who is 100% legit.
27 comments
[ 3.9 ms ] story [ 76.2 ms ] threadI was charged once, with transforming images for projection onto a sphere through special lenses. I didn't know the lens transform, the manufacturer kept that secret. I did know that the intensity across the spherical surface was of uniform brightness - the manufacturer boasted about this. With that bound I could guess the transform, as each pixel of the source image would have to illuminate an equal surface area of sphere once projected.
The manufacturer advertised a service to do this transform on your image, but they wanted $10K for each run! Once I presented my insights and a little math, they capitulated and did it at cost. I never got to write my projection software in the end.
Could you give me some pointers on where I could start studying about the math involved? Are those topics usually covered by good books on computer graphics?
Schneider, P.J., Eberly, D.H., 2003. Geometric tools for computer graphics, Morgan Kaufmann Publishers
indispensable
Is it possible to make the computer guess the correct value with AI? I guess a method is to identify the artificial objects like walls and doors, and assume they have straight lines. The nice curved tops of the windows may be a problem. And probably there must be a special case for circles like the clock.
Something i am currently working on is using fisheye images of the sky to predict cloud motion and to then estimate solar PV power output in the short term i.e. one to five minutes out.
It sounds simple but it isn’t yet even a child can observe cloud motion and predict to some degree when the disc of the sun will be obscured. Part of the inaccuracy is in the distortion caused by the fisheye lens.
I would be interested in seeing how well this works for subjects at infinity i.e. a fisheye lens pointed at the sky.
https://kovasi.photo/
no connection to him other than as a customer.
By the way, printing on aluminum seems like a really nice way to mount art on the wall
You could fix bad profile photos and such without the need to upgrade to more expensive optics.
https://imgur.com/a/9TgW7qy
Also, shout out to the European Space Agency for releasing high quality Planetarium documentaries under the CC license [2].
[1] http://paulbourke.net/dome/ [2] https://www.eso.org/public/videos/archive/category/fulldome/...
For simpler fisheye and defisheye effects, I wrote this web based tool a few years ago which some may find useful!
https://jywarren.github.io/fisheyegl/example/
He showed me some photogramatery work he was working on, I was amazed at how smooth everything was running given the precision and having worked with highly detailed tins and pointclouds myself on that days high end hardware. I asked him what kernel/engine he was using. He casually said it was his own. Which (having done toy ray tracing in C and dabling in openGL) blew my mind but also made me realise it would be absolutely necessary in order to do this kind of work with the type of control he needed. Everything from the ground up.
One of the few academics/researchers I've met who is 100% legit.