But where "Minority Report" author Philip K. Dick enlisted psychics to predict crimes, DHS is betting on algorithms: it's building a "prototype screening facility" that it hopes will use factors such as ethnicity, gender, breathing, and heart rate to "detect cues indicative of mal-intent."
I missed that part on my first read. Rereading it makes me think that ethnicity would be used along with age, gender, height, and weight to find a baseline for the 'indicative' factors like heart rate. I agree that once any genetic factors are used as indicative factors we have crossed a dangerous line. I sincerely hope that this phrase in the article was simply worded poorly.
It's much more likely that these inputs will be used as features to train some ML algorithm, rather than that they will actually be used to say "if black then detain".
The limitations of machine learning aside, this could actually work quite well if trained properly, because it wouldn't raise the questions that human choices do about bias. Given sufficient information, a machine-learning algorithm can make decisions based on factors that humans would refuse to consider or would intentionally err on the side of not considering.
One problem I see with this is that these statistics involve a lot of human bias. If you train your classifier on crime statistics, it's not based on a 'ground truth' (which is never attainable in a criminal case), but an expert's, i.e. judge's/jury's, decision.
That would indeed produce incorrect training for a classifier of "who should we pull out of line for a search". For that, you'd want to train exclusively on the observable characteristics of the people pulled out of line for a search versus whether you found something.
you don't need any evidence as it is obvious that ethnicity is part of the feature vector being fed into the system. Even if we imagine unimaginable - that it (together with other highly correlating data like place of birth) is not fed into the system from the start, the face recognition and other input feature vectors generating modules would produce the configuration in the parameter space where basic generic clustering at the pre-processing stage would also naturally cluster along ethnicity lines thus still making ethnicity an input for higher levels of the system. How this input will play inside that giant neural/statistical net wouldn't be possible to say as debugging such nets is an endeavor in itself. Previous implementations of such a pre-crime detection neural nets - using homo sapiens brain as hardware - produced "being black is a probable cause", and nobody was able to debug it in sufficient detail. Would we be able to produce better results using silicon based hardware?
In it, the statement is made that the technology will use factors such as ethnicity and gender to determine who was thinking about committing a crime.
I wonder if tech like this will encourage more self-segregation? People, particularly blacks, will discover over time that they will encounter less legal trouble if they behave in a certain fashion. That includes staying away from places where 'people like you' would not be expected.
EDIT: OK ... I see your comment below now dkokelley.
Consider for a moment that even if they did this tomorrow and announced it on the front page of the Times, people would continue to live in America, pay taxes, and take commercial flights.
The time to leave was some time ago. The first, second, fourth, and fifth amendments are all gone, now.
I'm a little disappointed with the response here. This isn't "racial profiling".
FAST is designed to track and monitor, among other inputs, body movements, voice pitch changes, prosody changes (alterations in the rhythm and intonation of speech), eye movements, body heat changes, and breathing patterns. Occupation and age are also considered. A government source told CNET that blink rate and pupil variation are measured too.
This technology should only be used to say "look here" to assist the people behind the technology. Ideally, this will help a few screeners identify people in a crowd who are behaving abnormally. This 'tech' has been used in security for ages. Only earlier we called it "I have a funny feeling about that guy" and now we can say "this man is suspiciously nervous, and here's the science to show it".
I don't want to get political, but the politically correct method of pure random screening in airports is horribly ineffective. Security professionals in unstable regions know this, which is why they are trained to look for the exact things this device is looking for.
The danger of this technology isn't that it will start issuing arrest warrants based on probable intent. The danger is that the people behind the technology will get it wrong. History shows that we don't need technology to help get it wrong.
Please don't confuse this technology with political positions. Has the TSA gotten out of hand? Sure. Are our rights being violated? Probably in many cases. Does this technology have anything to do with that? That only depends on how it's used.
We can't unequivocally say it's not racial profiling. All we know from the article is that ethnicity is being used as a factor. If a person's race/ethnicity can in any way make it more likely for the system to flag that person, then racial profiling is arguably happening.
So the question is: Under this system, can a person's race increase the likelihood of a positive result? It seems the answer is yes. If race is a factor, that means it's somehow being used as in input into the algorithm that determines a person's suspicion score. Maybe it's not something as heavy handed as "boost the suspicion score if the person is of race X." Suppose it's "weigh eye movement more heavily than breathing if the person is of race X." Even then, the effect is to increase the likelihood of a positive result _for some people_ based on their race.
I missed the explicit part about ethnicity. Rereading that makes me think that those factors are used as a baseline along with age, height, and weight to determine the normal 'indicative' factors like pupil dilation and heart rate, and then flag people who are abnormal given their ethnic, age, height, and weight group norms.
It seems that either the phrase in question was poorly worded, or the DHS is in for a serious media storm.
The real problem is that it presupposes thought-crime. The DHS document in the sidebar says FAST will "identify individuals with malintent ... *Malintent: the state of mind of individuals intending to cause harm ..."
Whether it's really scientific, I doubt we'll ever find out. But people will be more likely to believe it's scientific because it comes out of a box that uses 'algorithms'.
Another problem is one that Schneier writes about: profiling leaves vulnerabilities that can be probed, discovered, and exploited. In the case of trying to sense malicious intent, it could be sending someone who doesn't know they're an attacker. Or someone who doesn't exhibit the outward signs that are being profiled.
The real problem is that it presupposes thought-crime.
The problem with that is that it's hysterical bullshit, and implies that people will be prosecuted for thoughts. This is a tool for helping to target effort in the screening process.
To be fair, the article itself was guilty of being nonsense first. Mentioning 'ethnicity' as an input to the system was the give-away. Even if true, this would never be admitted by whoever was developing the system in question. When I read that, the red flag went up, and I assumed the entire article was untrustworthy. But then, I wasn't on the lookout for sensational stuff to get worked up about.
I agree that a system like this could be probed and exploited. The real issue comes up after someone is flagged as suspicious. Is this person arrested right away? Is this person subject to additional screening? It would violate basic human decency to arrest someone for appearing suspicious, regardless of whether or not he or she was 'scientifically suspicious'. Taking a closer look at suspiciously behaving individuals doesn't seem to cross that line for me. We've been doing that for years. The only difference now is that we have a machine to help scale that effort.
This is definitely the key point of the issue. As long as people aren't arrested for 'intent' to commit a crime - which is really impossible to prove without action - this tech might help out a lot. If it works as advertised and is used responsibly, it might help bring certain areas closer to security, instead of security theater.
The privacy implications need to be managed carefully; it's probably too much to ask for them to not be botched.
The other problem is that there is a whole grey area between "let the guy go, he isn't doing anything" and "tackle and arrest him now". The person can be detain, held up, put in a different queue, screened, probed, strip searched, etc.
Given TSA's record so far, it will result in various situations which violate people's rights without ever arresting them.
I may be wrong, but it seems to me that this tech, just like the metal detector and the porno scanner, are just ways of giving less-than-expert staff the ability to perform expert-like security screenings. In other words, someone with a two week training course in how to fire up this magic box is now charged with making the determination of whether each person passing through their station needs to get a secondary screening (of various invasiveness). What is this employee to do when the machine tells him, "we've got to strip-search this person now" and the person is a 95 year old woman in a wheel chair? How about if it's a 5 year old?
Maybe we need to give the TSA security more training/hire more professional security staff that would use years of experience rather than magic tools to determine who could be dangerous. Or do what the is done in Mexican airports: you push a button and if it comes out red you get your bags searched; if green, you can proceed. Random number generator FTW!
it isn't about exploiting the system by some individual(s).
It is about society making a new step enabled by the technology:
>The real problem is that it presupposes thought-crime.
exactly. The modern theory of organized violence allows for the society to apply violence through its government as executive organ to exterminate (usually by making a law and enforcing it or by just acting directly like using Reaper/Hellfire) whatever the society deems a danger to itself. The only recognized limits on this violence is in the Constitution and there is nothing in, for example, the US Constitution about freedom of thought. Thus as soon as thoughts can be distinguished/categorized, some of them can be outlawed and the law enforced.
Btw, would it be a fair use to mentally replay the scenes from a movie? Only if it is limited to 30 sec? Or only if memorized/replayed resolution is below some threshold? ... We're in for an interesting ride.
The CNET article says it's a system that [DHS] hopes will use factors such as ethnicity, gender, breathing, and heart rate. In the CBS News article linked from there, essentially the same claim is made.
There's no actual evidence in either article that ethnicity will be used.
One of the documents on the EPIC site (fastinstallation.pdf) says "The system is gender-, culture- and age-neutral".
I've skimmed through all the documents linked from there and can't find anything that admits to using ethnicity or gender. (There is one paragraph that says that a FAST trial will collect ethnicity and gender information "to understand whether the technology performs differently with different demographic groups", which is a different thing entirely.)
If the system is ever deployed, I wouldn't be desperately surprised if it turns out to use ethnicity as evidence of malintent. But it does look as if Declan McCullagh may be making a dramatic claim for which he lacks evidence.
Least likely organization to pull this off in the world -- other than my grandmothers knitting group, but they have enough sense not to try doing things they will surely fail at.
What would happen to something like this, if it starts getting a lot of false positives. Considering it's breathing and heart rate, could we not easily make it give out false positives? I mean it would probably inconvenience us, as an individual, but after a few months of tons of false positives, would they not abort it?
Hate to play devil's advocate, but it sounds somewhat similar to what Israel does with their airports, but leveraging technology to identify all details (and more than a human could reasonably do), rather than training individuals to do the same.
Methods, deployment (in terms of location) and data retention are obviously the distinguishing factors here.
>Hate to play devil's advocate, but it sounds somewhat similar to what Israel does
who said that Israel is human rights, non-racial/ethnical-discrimination, etc... champion?
Don't get me wrong - i'm not anti-Israel, i completely understand the necessity of the war they fighting for their survival. It is just that when you fight a war for your survival you don't have the luxury of championing human rights. Israel situation isn't an example to follow, it is a problem to be fixed when the war is over.
These "precautionary" measures and profiling databases are being developed in most parts of the world. Governments are becoming more and more intrusive in personal lives in the name of security. I really wish someone would buy some land or islands and create a modern country based on libertarian principles, it will be an interesting experiment/startup, although I am not sure if you can start a country just by buying land.
Governments consist of the same people like you and me. Your imaginary "modern country based on libertarian principles" would consist of the same people and it will look the same (if not worst) after short time. Problem isn't government. Problem is "we, the people".
(my old country was the leader in implementing "modern country based on best principles known to the humanity at the time". The main mistake people do when look back is blaming the principles, saying that these principles were bad and these are different and better. Nope. After all, the best principles of yesterday - communism and fascism - looked very different (with each claiming to be the best), yet both produced the same result, and today we do understand how much they were the same. The best principles of today (libertarianism according to you, Sharia Islam according to several hundred millions of some other people, ...) will be superseded by the best principles of tomorrow. It isn't the principles that kill millions of other peoples and make lives of other people nightmare. It is other people who do it. It isn't modern country that we need, it is the modern people. For example, people who just wouldn't compile the big database, wouldn't run the analysis and wouldn't issue the order to act upon the result of analysis. It is just that simple and just that hard. )
44 comments
[ 3.1 ms ] story [ 86.4 ms ] threadThe deployment of this technology should not be acceptable to anybody who believes in our right to live in a free society of equals.
But where "Minority Report" author Philip K. Dick enlisted psychics to predict crimes, DHS is betting on algorithms: it's building a "prototype screening facility" that it hopes will use factors such as ethnicity, gender, breathing, and heart rate to "detect cues indicative of mal-intent."
In it, the statement is made that the technology will use factors such as ethnicity and gender to determine who was thinking about committing a crime.
I wonder if tech like this will encourage more self-segregation? People, particularly blacks, will discover over time that they will encounter less legal trouble if they behave in a certain fashion. That includes staying away from places where 'people like you' would not be expected.
EDIT: OK ... I see your comment below now dkokelley.
The time to leave was some time ago. The first, second, fourth, and fifth amendments are all gone, now.
If you tell someone, "The world is looking like Minority Report" they go "Cool!"
MR-world has cool cars, is clean, the govt seems to have only one flaw, and Tom Cruise will wander by and save the day.
FAST is designed to track and monitor, among other inputs, body movements, voice pitch changes, prosody changes (alterations in the rhythm and intonation of speech), eye movements, body heat changes, and breathing patterns. Occupation and age are also considered. A government source told CNET that blink rate and pupil variation are measured too.
This technology should only be used to say "look here" to assist the people behind the technology. Ideally, this will help a few screeners identify people in a crowd who are behaving abnormally. This 'tech' has been used in security for ages. Only earlier we called it "I have a funny feeling about that guy" and now we can say "this man is suspiciously nervous, and here's the science to show it".
I don't want to get political, but the politically correct method of pure random screening in airports is horribly ineffective. Security professionals in unstable regions know this, which is why they are trained to look for the exact things this device is looking for.
The danger of this technology isn't that it will start issuing arrest warrants based on probable intent. The danger is that the people behind the technology will get it wrong. History shows that we don't need technology to help get it wrong.
Please don't confuse this technology with political positions. Has the TSA gotten out of hand? Sure. Are our rights being violated? Probably in many cases. Does this technology have anything to do with that? That only depends on how it's used.
So the question is: Under this system, can a person's race increase the likelihood of a positive result? It seems the answer is yes. If race is a factor, that means it's somehow being used as in input into the algorithm that determines a person's suspicion score. Maybe it's not something as heavy handed as "boost the suspicion score if the person is of race X." Suppose it's "weigh eye movement more heavily than breathing if the person is of race X." Even then, the effect is to increase the likelihood of a positive result _for some people_ based on their race.
It seems that either the phrase in question was poorly worded, or the DHS is in for a serious media storm.
Personally, I'm going to need more evidence than "an article on the Internet makes an outrageous claim, therefore the outrageous claim is true".
Whether it's really scientific, I doubt we'll ever find out. But people will be more likely to believe it's scientific because it comes out of a box that uses 'algorithms'.
Another problem is one that Schneier writes about: profiling leaves vulnerabilities that can be probed, discovered, and exploited. In the case of trying to sense malicious intent, it could be sending someone who doesn't know they're an attacker. Or someone who doesn't exhibit the outward signs that are being profiled.
The problem with that is that it's hysterical bullshit, and implies that people will be prosecuted for thoughts. This is a tool for helping to target effort in the screening process.
To be fair, the article itself was guilty of being nonsense first. Mentioning 'ethnicity' as an input to the system was the give-away. Even if true, this would never be admitted by whoever was developing the system in question. When I read that, the red flag went up, and I assumed the entire article was untrustworthy. But then, I wasn't on the lookout for sensational stuff to get worked up about.
Given TSA's record so far, it will result in various situations which violate people's rights without ever arresting them.
I may be wrong, but it seems to me that this tech, just like the metal detector and the porno scanner, are just ways of giving less-than-expert staff the ability to perform expert-like security screenings. In other words, someone with a two week training course in how to fire up this magic box is now charged with making the determination of whether each person passing through their station needs to get a secondary screening (of various invasiveness). What is this employee to do when the machine tells him, "we've got to strip-search this person now" and the person is a 95 year old woman in a wheel chair? How about if it's a 5 year old?
Maybe we need to give the TSA security more training/hire more professional security staff that would use years of experience rather than magic tools to determine who could be dangerous. Or do what the is done in Mexican airports: you push a button and if it comes out red you get your bags searched; if green, you can proceed. Random number generator FTW!
>The real problem is that it presupposes thought-crime.
exactly. The modern theory of organized violence allows for the society to apply violence through its government as executive organ to exterminate (usually by making a law and enforcing it or by just acting directly like using Reaper/Hellfire) whatever the society deems a danger to itself. The only recognized limits on this violence is in the Constitution and there is nothing in, for example, the US Constitution about freedom of thought. Thus as soon as thoughts can be distinguished/categorized, some of them can be outlawed and the law enforced.
Btw, would it be a fair use to mentally replay the scenes from a movie? Only if it is limited to 30 sec? Or only if memorized/replayed resolution is below some threshold? ... We're in for an interesting ride.
The CNET article says it's a system that [DHS] hopes will use factors such as ethnicity, gender, breathing, and heart rate. In the CBS News article linked from there, essentially the same claim is made.
There's no actual evidence in either article that ethnicity will be used.
One of the documents on the EPIC site (fastinstallation.pdf) says "The system is gender-, culture- and age-neutral".
I've skimmed through all the documents linked from there and can't find anything that admits to using ethnicity or gender. (There is one paragraph that says that a FAST trial will collect ethnicity and gender information "to understand whether the technology performs differently with different demographic groups", which is a different thing entirely.)
If the system is ever deployed, I wouldn't be desperately surprised if it turns out to use ethnicity as evidence of malintent. But it does look as if Declan McCullagh may be making a dramatic claim for which he lacks evidence.
Security theatre works because people believe it works.
Methods, deployment (in terms of location) and data retention are obviously the distinguishing factors here.
http://news.ycombinator.com/item?id=1024850 for an article on it, from around two years ago.
who said that Israel is human rights, non-racial/ethnical-discrimination, etc... champion?
Don't get me wrong - i'm not anti-Israel, i completely understand the necessity of the war they fighting for their survival. It is just that when you fight a war for your survival you don't have the luxury of championing human rights. Israel situation isn't an example to follow, it is a problem to be fixed when the war is over.
(my old country was the leader in implementing "modern country based on best principles known to the humanity at the time". The main mistake people do when look back is blaming the principles, saying that these principles were bad and these are different and better. Nope. After all, the best principles of yesterday - communism and fascism - looked very different (with each claiming to be the best), yet both produced the same result, and today we do understand how much they were the same. The best principles of today (libertarianism according to you, Sharia Islam according to several hundred millions of some other people, ...) will be superseded by the best principles of tomorrow. It isn't the principles that kill millions of other peoples and make lives of other people nightmare. It is other people who do it. It isn't modern country that we need, it is the modern people. For example, people who just wouldn't compile the big database, wouldn't run the analysis and wouldn't issue the order to act upon the result of analysis. It is just that simple and just that hard. )
The Palestinians tried this recently. Heck, they didn't even need to buy the land; they already owned it. Hasn't gone too well for them yet.