I was tempted to downvote, but this is a legitimate question, and I guess you haven't had opportunity to learn about how bogus "lie detection" in general is. I happened to attend university where some of the earliest rigorous researchers on lie detection were active,
so people in my town have known for a long time that purported lie detection tests don't work, and indeed our state has long banned the use of polygraphs or other purported lie detection devices for any courtroom purpose. Law enforcement investigators in our state use actual interviews of witnesses and suspects, and by state law all police interrogations must be videotaped. That helps immensely in reducing mistaken conclusions from improperly conducted interrogations.
I'm actually thinking if it might be a good tool to detect distress in a subject. Not lying exactly, but might provide insight into a subject's emotional state non-invasively during questioning or interrogation. People might become distressed for many reasons that may not just be simple lying.
Perhaps even a Voight-Kampff-like test a la Blade Runner/Do Androids Dream of Electric Sheep?
Androids in that case were fully human, just manufactured. The principle differences were improved physiology and durable mental processes, which could have occurred naturally.
Its quite true that polygraphs and similar means of telling whether a person is distressed are almost useless. However, there have recently been some advances in using fMRI to do direct lie detection by looking at what parts of a person's brain are being used at a given time. This doesn't seem to be reliable enough for courtroom use yet, but its getting there.
I said it above, but to carry on with what tokenadult said; Lie detection is a little more complicated than detecting facial expressions. If you have a defensive, introverted person looking downward, they might be lying or that could be how he responds to whatever interrogation they are under. Conversely, a mentally distressed individual (or someone who has done a heavy amount of NLP) could lie straight to your face because they personally believe they are telling the truth.
This type of technology will no doubt be used to attempt lie detection, but it won't work. The only thing we can do is use the context clues of the face to infer what we think is going on.
I like how this kind of turns the "main idea" of computer vision on its head. Namely, instead of trying to emulate and catch up to human abilities, it augments them the way only a computer can; it plays to computers' strengths instead of their weaknesses.
Also, who would have thought you can determine someone's pulse just by looking at the color change of the face due to blood rushing in and out? If asked beforehand, I wouldn't have even said it was theoretically (physically) possible.
I also found it very surprising that cameras could pick these things up, there's actually an iPhone app from Philips that claims to be able to be able to read heart-rate, I assumed it was snake-oil at first :
I was skeptical too, but when I tried this app a few months ago, I compared it to my Garmin heart rate monitor at rest and after exercise - they were usually within 1bpm of each other. I was very impressed.
There is so much faint and indirect information in the world it's dumbfounding. We're crossing a threshold where computation can make this visible. It's really exciting.
Yes, it's hard to believe, but it's really possible to sense the small color changes due to blood flow in the face using cameras. My earlier work described how you just need a simple webcam :)(http://web.mit.edu/newsoffice/2010/pulse-camera-1004.html). As a result, I'm working on Cardiio (http://cardiio.com) to bring this technology to both the web and mobile very soon. If you're interested in computer vision based biosensing, please sign up!
Hmm, when I tried to sign up I got a message that I was already on the waiting list.
EDIT: It looks like the page is making a request to cardiio.com, specifically, and if you’re at www.cardiio.com (the one you linked to) the response gets blocked by Same Origin Policy, and the page treats it the same as a you’re-already-on-the-list response. You might want to avoid hardcoding the hostname in the JS, and/or force everyone over to the www or non-www domain.
But from a scientific point of view, you don't really need spatial resolution for the heartbeat example, though, do you? If you just spent the same amount of money on getting high temporal and spectral resolution, surely one could do very cool things with 'just one pixel', so to speak. Especially if you had say 16 different wavelengths, instead of just 3.
Congratulations for, what seems to me, opening up the door [1] to a new world of CV applications! And of course for having the courage to commercialize it while your academic career is taking off.
Do you think it would be possible to see shifts in photosynthetic activity in plants due to low/high nutrients/humidity? Especially when IR-filter removed?
To measure the pulse of a person, you need to consider an interval of a few seconds, so it's safe to assume that the only thing that the position and strength of the light are almost constant. The time that takes the plants to reduce the amount of nutrients or water is much bigger, so I think that it would be much more difficult to compensate the changes in the ambient light.
Yes. My father does research in this area. Specifically, inferring the health of vegetation (among other attributes of land cover) from statistical analysis of satellite imagery.
From what I recall, you can measure the level of photosynthetic activity in an area given the reflectivity of the land (and, I would think, some basic model of what type of vegetation is present in the area)
One example from his research: the fall of the Soviet Union had a statistically significant impact on weather patterns in Kazakhstan (the breadbasket of the USSR) because the land usage changed so much with agricultural policies before and after the fall.
that's right. what you're looking for is the transient changes (AC) in color due to increase blood perfusion whereas a person's baseline skin color can be considered the DC component.
Pulse oximeters work by looking at the fingernails, and they're the usual way to measure a pulse. There's a lot of blood flow in the face - try bumping into something sharp - so this isn't too surprising.
This other algorithm amplifies only motion, and the algorithm is less elegantly simple, however its results are very interesting and fun to watch as well. Make sure to watch the video at the bottom of the page
I was thinking about the implications of using this technique to analyze e.g. political speeches and try to catch people lying on the act. Your application (winning on card games) seems very interesting too.
Coincidentally so was I! With a person's microgestures amplified, you'd at least notice inherent changes in a person's physical behavior. Now, we need to remember that lying isn't as simple as "he's looking down and to the right" because that can just be a personality trait of the person.
With public figures, however, if you collect enough video of various activities, you could determine a baseline physical demeanor. Then, if the figure in question radically steps out of that baseline, you can send off red flags.
Ideally, this could be used to analyze facial dynamics to help determine focus, expression (for example, the SOEmote video that was recently posted), mood (though we now get back into personality traits), or, as the video showed, subtle changes in the physical body.
I was under the impression that micro-expressions (I assume you're referring to them) were very short in duration rather than in magnitude. So this would be the wrong kind of amplification.
micro-expressions .. were very short in duration rather than in magnitude. So this would be the wrong kind of amplification
This is exactly the kind of amplification you need. Something like the movement caused by mirror of an SLR camera (at the end of their demo) is very very short in duration, but they amplify the magnitude AND slow the video to allow it to be visible.
No, my point is that micro-expressions don't need to be amplified, because they are perfectly large in magnitude, just very short. So all you need is a freeze-frame.
Some amplification is still needed in my opinion, but you are correct. Amplifying the video of a high speed camera should aid in the some of the expression detection. Instead of something like eye movement, think something more subtle, like a person's brow being furrowed.
It seems like this technology can be used on historical videos. It'd be interesting to see the heart rates from past speeches where we know they lied, to see if there is a good correlation.
Until realize that even though the filters in this post have been applied to videos. It will be just as easy to apply the filters to audio, surely again resulting in a revealed heartbeat :)
Amazing for many reasons, but what I really find impressive is that messing with video encoding requires like a PHD in everything because of how complex things like H.264 are. At least that's my perspective after trying to grok the H.264 spec.
There are all kinds of optimizations and algorithms for network latency and video sizes, etc. I don't consider myself a layman but I certainly do not have the knowledge and skill to do anything at all with video.
Do you know what a Inverse sub-macroblock partition scanning process is? I don't, not even after reading the part about it in the spec.
a macroblock is a chunk of an image which is identified to have some constancy between frames. a sub-macroblock would be some subdivision of this for finer motion details. sub-Macroblock partitian scanning would be applying an algorithm to identify the sub-macroblocks in a set of images. Add the word "Inverse" though and now I have no idea what the hell that means.
This really doesn't have much to do with what makes computer vision difficult (or rather, interesting). I doubt very many computer vision researchers are conversent with the H.264 spec.
Typically toolkits like OpenCV take care of codecs for you (which make them quite annoyingly difficult to compile on different platforms).
(I speak as someone who had done commercial computer vision algorithm development.)
What precisely is your dissertation about? Inferring motion from compression artifacts? Exploiting the fact that many codecs have already done motion estimation so that you don't have to do image flow again?
I'm not seeing what the big deal is here. Extraction of differential information from video frames has been standard in video editing and compositing software for years. Grab a demo copy of After effects, folks.
This is kind of the stock-negative comment that I don't like about HN. It would be redeemed if it had a link, but instead its just a look-how-cool-I-am-that-I'm-not-impressed-by-this actually-pretty-awesome-technology-demo, dammit.
The commenter you are replying to (and downvoting) has done (IIRC) professional digital video work, and written many detailed and helpful comments on HN about digital media. The comment you're replying to isn't snarky. Assume good faith. Maybe he's just mistaken? Or maybe he's right, and the thread just doesn't like that?
Here are some best-of's to peruse before concluding that 'anigbrowl is part of the problem with HN:
* If a downvoted comment with no replies upsets you so much that you feel a need to pile on with a pointed negative comment, go to the bottom of the page, type the commenter's name in, click 'comments', then sort by 'score', and read the top 10 or so first. You may find the urge to pile on dissipates quickly.
* While it certainly isn't going to cause drama here ('anigbrowl being one of the more levelheaded contributors to HN, way more so than me), I think you'll find that comments like yours tend to read as personalizing disputes, and lead to embarrassing spats you'd be better off not attaching yourself to.
I referenced a particular piece of software, which I thought was well-known enough not to need a link. I am not trying to look cool, but rather point to existing cool technology, and stand by my comment.
"Naively, this sounds like one might (a) compute the translation from one pixel to the next in each frame, and (b) re-render the video with small motions amplified. Unfortunately, such an approach would lead to artificial transitions between amplified and unamplified pixels within a single structure. Most of the steps of motion magnification relate to reliably estimating motions, and to clustering pixels whose motions should be magnified as a group."
The math behind the grouping (recognizing the relevant area as an object, magnifying the motion of the area properly, and filling in the empty space intelligently) is the cool "computer vision" part.
I'll bite. I've used After Effects at the grab-a-demo-copy level for motion tracking and stabilization and rotoscoping. I don't remember anything that would have allowed me to detect and amplify color changes in someone's face that way, or amplify movement of a given period like the shots of someone's pulse or of a baby breathing -- especially not with a general-purpose temporal amplification filter instead of really specific code. I'd love to know how to do that stuff with After Effects, though. Can you be a little more specific?
My guess is you might end up with something along the lines of, say, take a specific clip of a guy's face, mask the face, average the color, detect the subtle changes with a script, and use the output of the script to control a color filter. Likewise if I had an existing clip of a particular sleeping baby, I could target the chest, motion track, and feed the tracking data into a filter. But that's a laborious way to fake particular outputs of what this software does in realtime on arbitrary clips of things moving or changing -- it's not nearly equivalent.
According to the paper, it is really quite a simple idea.
Imagine a filter that one might use in an audio context that boosts the amplitude of frequencies falling in a certain range. But now imagine applying that filter to a timeseries that actually comes from the intensity of a single pixel over time.
This has the effect of emphasizing motions that occur over timescales that correspond to the selected band of frequencies.
The reason this is true actually comes back to the Fourier transform -- you can imagine the motion of part of a complex image to be composed out of the linear combination of many simpler 'sinusoidal motions', each of which is being amplified by the per-pixel filter.
There are two streams of information being analyzed. First is color, observed as one position changing properties from one frame of video to the next. Second is position, which is a change in location of things within the field of view from one frame of video to the next.
Both color and position changes can happen at the same time, but it's easier to see how this information is used if you start off by treating them separately.
For color changes, imagine video of a stationary person's face where the center pixel of every picture is the center of the face. The color at that center pixel changes from frame to frame. At frame 1 the red value (of an RGB triplet) is maybe 200 (out of 255), and by frame 5 the red value is 203. This is a temporal red shift of just 3 levels over 5 frames.
For position changes, imagine video of a beach ball in a still pool of water, where the center of the ball changes from frame to frame. At frame 1, the position is (in x,y) (512,512), and at frame 5, the position is (512, 515). This is a spatial shift of just 3 pixels over 5 frames.
In both of these cases, the observed change is probably too smal for humans to notice in casual viewing, so an exaggeration of some kind is desired. To make the change noticeable, we'll take the change of signal (3 values over 5 frames) and "amplify" it. These modified ("amplified") values will be shown on top of the original source video, resulting in an exaggerated change. If a color shifted by 3 red levels, we can show that by varying the color by 30 red levels. If a position changed by 3 pixels, we can show it changing by 12 pixels. We can choose the level of amplification best suited for observing the changes we've isolated.
Now the really cool aspect of these data streams is that their resulting "amplifications" don't need to be applied to the same data stream that they were observed in. It's possible to take a position change, and represent it as a color change. It's possible to take a color change, and stretch it out over a longer time (over more video frames). Any combination of properties you like can be used to represent your observed set of changes. Some of the most interesting representations will be a mixture of different data streams, represented as another mixture of exaggerations to the original video.
Hm... I get the color one, and the position one in theory... However, if you have a real-life video, how do you determine where the "center of the ball" is? How do you even define what an object is, what are it's edges, and in which direction did it move?
This type of technology would be a nice novelty item to implement into a futuristic zombie story. For example, there could be a large battle between humans and zombies, and the humans are equipped with pulse-magnifying glasses, allowing them to identify friend vs. foe.
This technology is badly needed today for skin cancer screenings. We could save tens of thousands of lives a year with better detection of cancers and pre-cancerous growths. It is crazy that most screenings in 2012 are still 3 minutes a year with a harried doctor.
If anybody could help clarify a concept (a specific filtering method), from video:
"we filter the sequence temporally using a "???" ...and boost the temporal signal to generate a video..."
(at 2min. 45sec in video)
What type of filter is being described (missing from quotes; "???") ? Bonus upvotes for providing a URL to definition (detailed description of the missing term).
He talks about a "Temporal Bandpass Filter". Not sure about the "temporal" part, but in DSP a bandpass filter is a filter (think Audio-EQ) that passes frequencies within a specific range and rejects the rest. See https://en.wikipedia.org/wiki/Bandpass_filter
It would seem to me from perusing the spec that they are simply taking a video, doing a 1D DCT on the time axis and amplifying a "frame" in the result corresponding to the desired period. Hopefully someone can correct me if there's more to it.
I'd like to get at one of their sample videos before filtering because it would be pretty easy to toss together a workalike based on the above.
There's a whole business to be made off that five second clip where they amplify the motion of the sleeping baby's chest.
Anybody who's ever been a parent can remember going up to check on the baby at night, standing over the crib for a full minute thinking "move! Move! MOVE!" at the lifeless body of their baby, running all the horror stories they've heard about cot death over and over in their head until finally the kid's chest moves enough to breathe a sigh of relief and go back to bed.
The first company that adds this feature to a video baby monitor will make a mint.
There's a bit difference between building a business that preys on the fears of a new parent and a business that acknowledges that new parents already have fears and tries to reduce them. This I would say falls firmly into the later category.
It's the difference between a tool which helps you make sure that your baby is ok and one which tries to persuade you that if you don't use it you're creating unconscionable risks to your baby which this product could save you from. The former is a tool which is good, the later is one which preys on parents.
Totally disagree. The two devices are one and the same.
Checking on your baby in the middle of the night because you woke with a sudden fear - normal parent anxiety, makes a funny story later.
Providing a device to allow constant electronic monitoring - validating an irrational fear, reinforces anxiety, feeds into an obsessive loop, invents issues for new parents just when they need to be calm and rational.
I'm very much of the opinion that anything which causes peace of mind is good thing and teaches the parents that their fears aren't grounded in reality. I'd say that most the fears that new parents have are rational in the overarching sense (in that it's entirely rational for a parent to worry about about their baby's health whenever the baby seems to deviating from the expected), but new parents lack the knowledge to determine if any one particular event is something that constitues an actual emergency, or is perfectly normal. Babies commonly have minor health problems (such as colic or jaundice) when they're very young which don't require medical attention, but sometimes they do require attention and it can be hard to tell which is which since babies only have very limited ways to express that they're suffering. A tool which provides more information to the parents (like, say, being able to see that the baby's heart rate or respiration has dramatically changed and so they might not be crying as a result of colic, but because there's something else wrong) and let's them make a more informed decision is clearly of benefit and is by no means 'validating an irrational fear'.
Peace of mind is most often effected by turning the mind from obsessive loops. Selling expensive gadgets to nervous parents is way too tempting a profit center to be responsibly marketed. How about by prescription only? That would be a reasonable compromise. Ideally someone objective could be put into picture, to prevent abuse.
Lists some. Right after saying we don't know what causes it. The things listed are pretty generic, like smoking around a mother expecting, etc. But nothing a hundred percent. It's also the biggest cause of death in children under one year of age.
Not sure where you are getting that it's something not to worry about. I do wonder how many children you've had though.
Edit: Oh the point's made above were to beg the how would one get prescribed something that so far seems pretty random.
Oh, yeah, good point, attack me instead of the argument when you run out of things to say.
I have 3 kids. I trusted in fate to take care of things I had no control over, and was lucky they lived to be healthy confident adults.
I know of super-careful parents that raised precious children frightened of the world. That loses something too - like the purpose of living.
I know engineers want to solve everything with an engineering solution. That way leads madness. Next we'll be sticking monitors in the womb, because, you know, sometimes the foetus doesn't develop right and maybe we can do something about that too.
I'm sorry no. The expression of "preying on parents fears/anxiety as marketing a device" is not literal. It is about drumming up those fears at the time of the sale, not at the time fo the use. In short, if I make no claims about making the motivator of your fears go away, but I do make claims about helping you manage the fears themselves, that isn't preying on your fears and anxiety. I believe the technical term for that is "helping".
The obsessive loop is, "maybe my baby isn't breathing <-> check on baby <-> it's breathing <-> go back <-> maybe my baby isn't breathing". If you can go right from, "maybe my baby isn't breathing <-> of course not, this idiot box would tell me if he/she weren't" (and sure it's fair that that is far from certain), that's huge progress and it can deflate obsessive compulsive behaviour, anxiety and fear. It's the equivalent of giving a kid a stuff toy so they aren't scared in bed at night.
$5 for a suitable app is awfully cheap assurance. $50 for a dedicated device, ditto. We're not talking "taking advantage" here, we're talking something every parent in history wanted.
What part of "the child is, in fact, not breathing" isn't obviously actionable?
Seems you've never stared at a child in dim light, waiting for some sign of life, knowing everything is probably ok and you don't want to disturb the snowflake to find out, but know that there are nontrivial odds of spontaneous expiration and the consequence of further inaction may be dire. Cheap insurance, peace of mind, good profits in mitigating a fair concern.
I'm asking how this "not breathing indicator" could be used to improve outcomes. What does one do if it signals a problem--are parents going to be trained to perform CPR on their infants (and would that even help if done properly)?
Pandering to parents' fears about something they can do nothing to solve does not seem to me like a productive thing.
:( unfortunately might not be entirely applicable to this problem... unless some measure of blood density and hence probably glucose mix... but at least at this layer of filtering and reasoning, it appears to be about minute amounts of motion... is there something about motion that could denote glucose? trembling maybe? maybe a score could be derived using something that borrows from this technique, but you'd have to put more into it than it seems this technique specifically shows.
I was thinking more skin color or possibly changes in the eye (if I remember right eyes get swollen when blood sugar is high) Honestly, I don't know if there are enough outwardly visible changes for this to even be possible.
But there's a lot of money out there waiting on the person who can figure out a way to use a cell phone do something like this.
Good idea, but apparently much money has been spent on it already. Excellent article on the physics of all things non-invasive (but especially glucose): http://www.mendosa.com/noninvasive_glucose.pdf
The simplicity and elegance of this algorithm is really breathtaking. I'm mostly thinking out loud here, but I think you could use a related process to do inter-frame interpolation for smooth time-scaling of video.
This was made by resampling in time with a DCT. Problems with this: 1. practically, it needs the entire movie in memory. 2. gibbs phenomenon in time is really weird. These could both be addressed by using short overlapping windows, and then the process becomes just like typical sinc/lanczos interpolation in time.
Well, yes, you can interpolate in the frequency domain like that, which works pretty well for resampling 2D data as well. However, it does not transition shapes from location to location; instead, it looks like one frame is fading into another. What's fascinating about this multiscale algorithm is that it appears to actually exaggerate the motions rather than merely increasing the contrast of the changes.
From my experience with CV, one of the biggest unsolved problems and rarely publicized aspect of the field is how much variable tuning is often required to get good results. I'm curious how sensitive these demos are to different environmental conditions and the robustness of the algorithm.
I have a tangential question: What's the state of technology that would allow one to see through the eyes of another? This would be useful for piano tutorials, etc. I think it could be done with a separate eye-tracking device and a camera located between the eyes.
Not necessarily; I usually don't use a score when I play. It would also be insightful to see the 1st person perspective of a jazz musician playing from a lead sheet, to see how his eyes jump between page and piano.
I just downloaded an app[1] that measures heartbeat rate by taking video of a fingertip.
I was stunned by the brilliance of the application, how ubiquitous video sensors can replace other senses.
There's something else to discuss here: We have a recently published paper from MIT, including mathematical details of the process; and at the same time a stack of closed-source commercial applications already in the market with slick interfaces. Are they using the same technique? Is one app-maker trawling preprints for app ideas? Are many app-makers copying one pioneering app-maker?
133 comments
[ 3.2 ms ] story [ 189 ms ] threadI was tempted to downvote, but this is a legitimate question, and I guess you haven't had opportunity to learn about how bogus "lie detection" in general is. I happened to attend university where some of the earliest rigorous researchers on lie detection were active,
http://www.fas.org/blog/secrecy/2006/09/david_lykken_and_the...
http://images.salon.com/april97/news/news970410.html
http://antipolygraph.org/articles/article-018.shtml
http://www.psychologicalscience.org/observer/getArticle.cfm?...
so people in my town have known for a long time that purported lie detection tests don't work, and indeed our state has long banned the use of polygraphs or other purported lie detection devices for any courtroom purpose. Law enforcement investigators in our state use actual interviews of witnesses and suspects, and by state law all police interrogations must be videotaped. That helps immensely in reducing mistaken conclusions from improperly conducted interrogations.
Perhaps even a Voight-Kampff-like test a la Blade Runner/Do Androids Dream of Electric Sheep?
This type of technology will no doubt be used to attempt lie detection, but it won't work. The only thing we can do is use the context clues of the face to infer what we think is going on.
Also, who would have thought you can determine someone's pulse just by looking at the color change of the face due to blood rushing in and out? If asked beforehand, I wouldn't have even said it was theoretically (physically) possible.
http://itunes.apple.com/us/app/vital-signs-camera-philips/id...
Hmm, when I tried to sign up I got a message that I was already on the waiting list.
EDIT: It looks like the page is making a request to cardiio.com, specifically, and if you’re at www.cardiio.com (the one you linked to) the response gets blocked by Same Origin Policy, and the page treats it the same as a you’re-already-on-the-list response. You might want to avoid hardcoding the hostname in the JS, and/or force everyone over to the www or non-www domain.
But from a scientific point of view, you don't really need spatial resolution for the heartbeat example, though, do you? If you just spent the same amount of money on getting high temporal and spectral resolution, surely one could do very cool things with 'just one pixel', so to speak. Especially if you had say 16 different wavelengths, instead of just 3.
But in the real world scenarios you want to take a look at more than one pixel, and use the rest of the picture for noise reduction & calibration.
That is for heartbeat. But yes, with better spectral resolution you might get some other cool data.
[1] famous illustration from the enlightenment: http://www.lehrer.uni-karlsruhe.de/~za874/homepage/aufklaeru...
From what I recall, you can measure the level of photosynthetic activity in an area given the reflectivity of the land (and, I would think, some basic model of what type of vegetation is present in the area)
One example from his research: the fall of the Soviet Union had a statistically significant impact on weather patterns in Kazakhstan (the breadbasket of the USSR) because the land usage changed so much with agricultural policies before and after the fall.
Quick link to the research overview page: http://globalmonitoring.sdstate.edu/research.php
https://play.google.com/store/apps/details?id=com.macropinch...
There's a red/infrared led on one side of the finger, a detector on the other side, and it measures the amount of light absorbed by the blood.
http://people.csail.mit.edu/celiu/motionmag/motionmag.html
This other algorithm amplifies only motion, and the algorithm is less elegantly simple, however its results are very interesting and fun to watch as well. Make sure to watch the video at the bottom of the page
http://people.csail.mit.edu/celiu/motionmag/VIDEO_siggraph_2...
Imagine that plugged into Google Glass. Overlay someone's heart rate next to their face while you're talking to them!
With public figures, however, if you collect enough video of various activities, you could determine a baseline physical demeanor. Then, if the figure in question radically steps out of that baseline, you can send off red flags.
Ideally, this could be used to analyze facial dynamics to help determine focus, expression (for example, the SOEmote video that was recently posted), mood (though we now get back into personality traits), or, as the video showed, subtle changes in the physical body.
This is exactly the kind of amplification you need. Something like the movement caused by mirror of an SLR camera (at the end of their demo) is very very short in duration, but they amplify the magnitude AND slow the video to allow it to be visible.
I imagine the drug atenolol would become popular among politicians.
It could be interesting to look over old films of riots to see how they may have been instigated...
Politicians in front of cameras are too heavily covered in face powder for their real color to shine through.
Of course, use of those techniques will themselves come to indicate guilt...
I wouldn't make assumptions. There's a valid point in wanting to defend your thoughts and intentions.
The technology is probably not mature enough for video analysis, though.
Does it work on cold-blooded animals too?
And personally, I don't think any of the "modern-day lies" that people tell would cause any kind of increase in heart rate.
it is not as simple as you wish. and that is good!
There are all kinds of optimizations and algorithms for network latency and video sizes, etc. I don't consider myself a layman but I certainly do not have the knowledge and skill to do anything at all with video.
Do you know what a Inverse sub-macroblock partition scanning process is? I don't, not even after reading the part about it in the spec.
Typically toolkits like OpenCV take care of codecs for you (which make them quite annoyingly difficult to compile on different platforms).
(I speak as someone who had done commercial computer vision algorithm development.)
Here are some best-of's to peruse before concluding that 'anigbrowl is part of the problem with HN:
* http://news.ycombinator.com/item?id=1875412 (MIDI)
* http://news.ycombinator.com/item?id=995096 (3D film)
* http://news.ycombinator.com/item?id=4011770 (Google Moog doodle)
* http://news.ycombinator.com/item?id=1263289 (DV on DSLRs)
A couple suggestions:
* If a downvoted comment with no replies upsets you so much that you feel a need to pile on with a pointed negative comment, go to the bottom of the page, type the commenter's name in, click 'comments', then sort by 'score', and read the top 10 or so first. You may find the urge to pile on dissipates quickly.
* While it certainly isn't going to cause drama here ('anigbrowl being one of the more levelheaded contributors to HN, way more so than me), I think you'll find that comments like yours tend to read as personalizing disputes, and lead to embarrassing spats you'd be better off not attaching yourself to.
Looking back, I think what riled me was the word 'folks'. As in, "nothing to see here, folks", although he didn't actually say that.
"Naively, this sounds like one might (a) compute the translation from one pixel to the next in each frame, and (b) re-render the video with small motions amplified. Unfortunately, such an approach would lead to artificial transitions between amplified and unamplified pixels within a single structure. Most of the steps of motion magnification relate to reliably estimating motions, and to clustering pixels whose motions should be magnified as a group."
The math behind the grouping (recognizing the relevant area as an object, magnifying the motion of the area properly, and filling in the empty space intelligently) is the cool "computer vision" part.
My guess is you might end up with something along the lines of, say, take a specific clip of a guy's face, mask the face, average the color, detect the subtle changes with a script, and use the output of the script to control a color filter. Likewise if I had an existing clip of a particular sleeping baby, I could target the chest, motion track, and feed the tracking data into a filter. But that's a laborious way to fake particular outputs of what this software does in realtime on arbitrary clips of things moving or changing -- it's not nearly equivalent.
Imagine a filter that one might use in an audio context that boosts the amplitude of frequencies falling in a certain range. But now imagine applying that filter to a timeseries that actually comes from the intensity of a single pixel over time.
This has the effect of emphasizing motions that occur over timescales that correspond to the selected band of frequencies.
The reason this is true actually comes back to the Fourier transform -- you can imagine the motion of part of a complex image to be composed out of the linear combination of many simpler 'sinusoidal motions', each of which is being amplified by the per-pixel filter.
There are two streams of information being analyzed. First is color, observed as one position changing properties from one frame of video to the next. Second is position, which is a change in location of things within the field of view from one frame of video to the next.
Both color and position changes can happen at the same time, but it's easier to see how this information is used if you start off by treating them separately.
For color changes, imagine video of a stationary person's face where the center pixel of every picture is the center of the face. The color at that center pixel changes from frame to frame. At frame 1 the red value (of an RGB triplet) is maybe 200 (out of 255), and by frame 5 the red value is 203. This is a temporal red shift of just 3 levels over 5 frames.
For position changes, imagine video of a beach ball in a still pool of water, where the center of the ball changes from frame to frame. At frame 1, the position is (in x,y) (512,512), and at frame 5, the position is (512, 515). This is a spatial shift of just 3 pixels over 5 frames.
In both of these cases, the observed change is probably too smal for humans to notice in casual viewing, so an exaggeration of some kind is desired. To make the change noticeable, we'll take the change of signal (3 values over 5 frames) and "amplify" it. These modified ("amplified") values will be shown on top of the original source video, resulting in an exaggerated change. If a color shifted by 3 red levels, we can show that by varying the color by 30 red levels. If a position changed by 3 pixels, we can show it changing by 12 pixels. We can choose the level of amplification best suited for observing the changes we've isolated.
Now the really cool aspect of these data streams is that their resulting "amplifications" don't need to be applied to the same data stream that they were observed in. It's possible to take a position change, and represent it as a color change. It's possible to take a color change, and stretch it out over a longer time (over more video frames). Any combination of properties you like can be used to represent your observed set of changes. Some of the most interesting representations will be a mixture of different data streams, represented as another mixture of exaggerations to the original video.
Baby monitors should probably be IR based; then this would appear to be a great addition allowing a pulse to be determined from the video feed.
Current baby monitors use a pad underneath to detect movement including breathing.
Personally, we used the "you can hear them screaming half-a-mile away" method of baby monitoring.
https://play.google.com/store/apps/details?id=si.modula.andr...
"we filter the sequence temporally using a "???" ...and boost the temporal signal to generate a video..." (at 2min. 45sec in video)
What type of filter is being described (missing from quotes; "???") ? Bonus upvotes for providing a URL to definition (detailed description of the missing term).
I'd like to get at one of their sample videos before filtering because it would be pretty easy to toss together a workalike based on the above.
Anybody who's ever been a parent can remember going up to check on the baby at night, standing over the crib for a full minute thinking "move! Move! MOVE!" at the lifeless body of their baby, running all the horror stories they've heard about cot death over and over in their head until finally the kid's chest moves enough to breathe a sigh of relief and go back to bed.
The first company that adds this feature to a video baby monitor will make a mint.
It's the difference between a tool which helps you make sure that your baby is ok and one which tries to persuade you that if you don't use it you're creating unconscionable risks to your baby which this product could save you from. The former is a tool which is good, the later is one which preys on parents.
Checking on your baby in the middle of the night because you woke with a sudden fear - normal parent anxiety, makes a funny story later.
Providing a device to allow constant electronic monitoring - validating an irrational fear, reinforces anxiety, feeds into an obsessive loop, invents issues for new parents just when they need to be calm and rational.
I'm very much of the opinion that anything which causes peace of mind is good thing and teaches the parents that their fears aren't grounded in reality. I'd say that most the fears that new parents have are rational in the overarching sense (in that it's entirely rational for a parent to worry about about their baby's health whenever the baby seems to deviating from the expected), but new parents lack the knowledge to determine if any one particular event is something that constitues an actual emergency, or is perfectly normal. Babies commonly have minor health problems (such as colic or jaundice) when they're very young which don't require medical attention, but sometimes they do require attention and it can be hard to tell which is which since babies only have very limited ways to express that they're suffering. A tool which provides more information to the parents (like, say, being able to see that the baby's heart rate or respiration has dramatically changed and so they might not be crying as a result of colic, but because there's something else wrong) and let's them make a more informed decision is clearly of benefit and is by no means 'validating an irrational fear'.
Lists some. Right after saying we don't know what causes it. The things listed are pretty generic, like smoking around a mother expecting, etc. But nothing a hundred percent. It's also the biggest cause of death in children under one year of age.
Not sure where you are getting that it's something not to worry about. I do wonder how many children you've had though.
Edit: Oh the point's made above were to beg the how would one get prescribed something that so far seems pretty random.
I have 3 kids. I trusted in fate to take care of things I had no control over, and was lucky they lived to be healthy confident adults.
I know of super-careful parents that raised precious children frightened of the world. That loses something too - like the purpose of living.
I know engineers want to solve everything with an engineering solution. That way leads madness. Next we'll be sticking monitors in the womb, because, you know, sometimes the foetus doesn't develop right and maybe we can do something about that too.
The obsessive loop is, "maybe my baby isn't breathing <-> check on baby <-> it's breathing <-> go back <-> maybe my baby isn't breathing". If you can go right from, "maybe my baby isn't breathing <-> of course not, this idiot box would tell me if he/she weren't" (and sure it's fair that that is far from certain), that's huge progress and it can deflate obsessive compulsive behaviour, anxiety and fear. It's the equivalent of giving a kid a stuff toy so they aren't scared in bed at night.
$5 for a suitable app is awfully cheap assurance. $50 for a dedicated device, ditto. We're not talking "taking advantage" here, we're talking something every parent in history wanted.
IMO a more likely outcome than preventing SIDS is a device malfunction leading to parental heart trouble and general distress.
Perhaps an analogue would be a device that illuminates a green light when there is not a (dreadful disease|nuclear strike|zombie horde) incoming...
To be honest, a soothing green light that indicates a bunch of dangers are not to worry about sounds pretty good to me too.
I think a device malfunction would at worst lead to a baby being forced awake by a worried parent to check everything is ok...
What part of "the child is, in fact, not breathing" isn't obviously actionable?
Seems you've never stared at a child in dim light, waiting for some sign of life, knowing everything is probably ok and you don't want to disturb the snowflake to find out, but know that there are nontrivial odds of spontaneous expiration and the consequence of further inaction may be dire. Cheap insurance, peace of mind, good profits in mitigating a fair concern.
Pandering to parents' fears about something they can do nothing to solve does not seem to me like a productive thing.
Call 911. Do CPR (yes, some of us are trained). Get in the car and race the kid into the ER. Clear the airway. Solve the problem.
What, you're going to SIT THERE WHILE YOUR CHILD DIES???
This was made by resampling in time with a DCT. Problems with this: 1. practically, it needs the entire movie in memory. 2. gibbs phenomenon in time is really weird. These could both be addressed by using short overlapping windows, and then the process becomes just like typical sinc/lanczos interpolation in time.
http://people.csail.mit.edu/celiu/motionmag/motionmag.html
http://www.cs.columbia.edu/CAVE/projects/world_eye/
http://www.cs.columbia.edu/CAVE/projects/eyes_relight/
I was stunned by the brilliance of the application, how ubiquitous video sensors can replace other senses.
There's something else to discuss here: We have a recently published paper from MIT, including mathematical details of the process; and at the same time a stack of closed-source commercial applications already in the market with slick interfaces. Are they using the same technique? Is one app-maker trawling preprints for app ideas? Are many app-makers copying one pioneering app-maker?
[1] https://play.google.com/store/apps/details?id=com.macropinch... but there are many similar apps.
The mathematical model for the sensor is probably different, though.