Ask HN: Is there any way that using pinch-to-zoom could change/add pixels?
Context is here, talking about using an iPad to zoom in on video. [0] My initial thought is no, it just makes existing pixels bigger and blurrier. But then I saw this expert’s testimony[1] and I’m wondering if I’m missing something: “Wisconsin crime lab employee James Armstrong testified, under questioning from defense attorney Corey Chirafisi, that the software program adds pixels to the image and he cannot say with certainty what color the added pixels are.”
[0] https://arstechnica.com/tech-policy/2021/11/rittenhouse-trial-judge-disallows-ipad-pinch-to-zoom-read-the-bizarre-transcript/
[1] https://apnews.com/article/kyle-rittenhouse-technology-wisconsin-kenosha-homicide-b561bef68dc6aadaadc9b45a1bd93a19
20 comments
[ 2.5 ms ] story [ 56.6 ms ] threadStill a stretch, and quite a big one at that, so its probably not it.
In what context did this show up in the Rittenhouse trial?
What a show, I can see why they broadcast it all over the place.
At first I thought the same thing, but I'm not sure about that. It seems the main point is that the prosecution is required to prove that the manipulation by digital zoom doesn't alter the image in any other way. That seems reasonable. If Apple has a patent, then the methods should be explained enough to show exactly what is being manipulated. If it's not patented/documented, then I think it's reasonable to question what is going on within the black box.
Federal Rule of Evidence 1001 (e): A “duplicate” means a counterpart produced by a mechanical, photographic, chemical, electronic, or other equivalent process or technique that accurately reproduces the original.
https://www.law.cornell.edu/rules/fre/rule_1002
Federal Rule of Evidence 1002: An original writing, recording, or photograph is required in order to prove its content unless these rules or a federal statute provides otherwise.
https://www.law.cornell.edu/rules/fre/rule_1003
Federal Rule of Evidence 1003: A duplicate is admissible to the same extent as the original unless a genuine question is raised about the original’s authenticity or the circumstances make it unfair to admit the duplicate.
The defense argued that a zoomed-in video is not a duplicate under the rules because the zooming algorithm changed the pixels. If that is true, then the zoomed-in video can't be used as evidence and you have to use the original. If it is false, then the zoomed-in video can be used as evidence.
That's a real argument and the prosecution was unprepared to defend it. If you are going to blame someone (hint hint), don't blame the judge.
A question I didn't see talked about is did they think zooming in would help? My experience has generally been that digital zoom doesn't really help in most cases, especially for distant shots (which if looked like they were viewing).
(I don't think pinch-to-zoom adds sharpening, but it's something that could use an expert to double check.)
Now we zoom. We're displaying a smaller part of the source image, but putting it on the same number of destination pixels. So where do the extra pixels come from? There are (at least) two possible answers:
1. We repeat source pixels across multiple display pixels. This leads to aliasing - to blocky stair-steps in the displayed image.
2. We make up values for the extra display pixels that were not in the source image. This is done by interpolation, not just by random guessing. Bicubic interpolation is pretty good. But still, the program is in fact "making up" values for the new pixels.
Now each 3 pixels are being shown on 10 pixels. Each pixel is being spread across 3 1/3 display pixels. How does that work?
So say you're dealing with the pixel at (0, 1.333). You take the pixels at (0, 0), (0, 1), (0, 2), and (0, 3), and you run a cubic interpolation to find the value at 1.333. (You do this three times, for red, green, and blue).
If you're at non-integer coordinates in both directions, then you run an interpolation that is cubic in both directions (that is, a bi-cubic interpolation).
Some people can make paintings that look an awful lot like a photograph, that is, they have a mental model of what scenes look like and can construct an image from that model.
Computers can create photorealistic images using raytracing techniques and also with neural networks
https://deepai.org/machine-learning-model/text2img
It's very possible a scaling algorithm could guess at what is missing in the picture and fill something in. That doesn't mean that is going on with Apple products in 2021.
The answer is: bicubic, yes; bilinear: yes; nearest neighbor, no.
To try to answer the question, yes it does/could. Although it depends much on the interpolation algorithm used to enlarge the image or if there was another process that could produce significant alteration such as an AI assisted process.
If you use interpolation to "make it clearer" you're adding information to it and it stops being an accurate representation of the original. It might look clearer to you and it could help you gain more insight, but after the enhancement it's no longer an accurate representation of the original. The very meaning of the word interpolation is "the insertion of something of a different nature into something else."
As an example imagine there's a far away video of someone handing a phone to another person and you take the best frame you can for analysis. In the original it looks like that person blur is handing something to the other person but it's just a blackish line.
After processing with different algorithms in one it now looks like the person is pointing a gun, in the other shows an empty hand and the AI one for some reason shows it's a bag, and so on.
More specifically about the trial the expert witness declared that he does not actually know how the algorithms used work and that he didn't compare with the original. Which I personally find baffling because this is a laboratory that analyzes evidence that may decide the future of a lot of people's life. The technician needs to know how it works in order to be able to reach an objective conclusion that what they're presenting is accurate.
This could easily be used to digitally zoom in on an image, for example.
Where things went sideways here is the prosecution expecting to be able to use pinch-to-zoom as testimony in court, when they should have actually gone through the standard CSI process of producing a digital zoomed image.
Now, to be honest, I think they should have been able to use pinch-to-zoom. If they lose the case because of this problem, I hope they can get a mistrial declared and then go back with the proper procedure.
Otherwise, the prosecution just plain screwed themselves over.