68 comments

[ 3.0 ms ] story [ 127 ms ] thread
Scary developments when it comes to state control. I wonder what the rest of the EU is going to do [1] - we have to wait until February to find out.

[1] https://www.theguardian.com/technology/2020/jan/17/eu-eyes-t...

I just find it funny how the "concerned brexiters" that are always decrying EU regulations are curiously absent from these discussions about face recognition overreach.
The EU has funded research on mass surveillance that use facial recognition for decades.
Sounds pretty dystopian to me.

When talking with people they sometimes say 'I did nothing wrong, this is just for the bad guys'. But what people fail to see is that 'bad guys' is a pretty fluid concept and it depends on the nature of the current government and who decides which people are on the lists.

Looking at the current British government it isn't even that hard to imagine the wrong hands that this technology could get into and cause damage.

Who maintains the databases with 'wanted faces'? Imagining a some corrupt corporation that just puts their enemies on these lists, think union leaders, critical journalists, etc, again, isn't that hard.

Never mind that this tech, being tech, isn't perfect and there are probably false positives that lead to very annoying situations.

(comment deleted)
The converse scenario is also bad. Imagine an organized government subversion cell that is willing to harm civilians to further its objectives and is skilled at hiding in public (one doesn't have to try very hard given the nature of asymmetric warfare in the 21st century). Surely, we want the ability to locate their members as efficiently as possible?

Considering the dystopia 1984 predicted is a good idea, but one should also consider the failure mode of "Snow Crash" anarchic dystopia, where sufficiently-organized individuals use modern technology to harm innocent people and government cannot stop them. The mafia stories in the US stem from the era where city organized crime had firepower that out-gunned the police.

Whatever tools implemented need to avoid creating (terrorists/freedom fighters depending on your viewpoint). At least in the US, we're taught a culture that lauds those who fought against tryranny. If the government loses perceived legitimacy(say by suppressing the expression of first amendment rights to free speech) then there would likely be an increase in militias and home-grown terrorism.
Honestly, I'd rather not want that ability than risk a never-ending unescapable 1984 dystopia.
Neither of these scenarios are de-facto inescapable. And if we're talking avoidance of technologies because of risk: historical scholars point to the message-consolidating effects of radio and television as a technology that enabled the rise of fascism in Europe between WWI and WWII (Orwell alludes directly to this with the hate rallies in front of Big Brother's TV image). Should we ban TV?
No, I want locating those members to be hard. I understand human nature and we know that no one will watch the watchers. There are lots of avenues for tracking these groups but it requires real work. You have to infiltrate and investigate.

You are also missing something. It is really about civilians being harmed? How many people died last year from terrorism in the US or the UK? How many died because people were texting and driving? Or drunk driving? Should we track when you text and drive the same way? Should we have police sit outside of every bar testing people to prevent harm? I think we all prefer to have the presumption of innocence and to face consequences for our mistakes and crimes, not to be tracked in case we do.

I think the UK is providing a very useful example of how one watches the watchers. The government is accountable via elections and their power seems held in check pretty well.

> I want locating those members to be hard

I respect your opinion, but the people who have lost loved ones to "random" attacks don't generally share that view. Hell, part of the purpose of building a society and a government is to minimize that kind of personal violence against people.

I understand what you are saying but are the people who lost loved ones to drunk driving or text driving any more understanding? You could argue it is even more senseless violence. We could easily make car connected breathalyzers mandatory in all cars. This would save 10K lives in the US alone. Are those thousands of lives not worth it in your mind?

I agree that government has a role in protecting people. We are both free to vote for representatives that are either for spying on citizens or against spying on citizens. We can also move if the other side wins.

Are we arguing financial cost or privacy cost?

I personally think the financial burden of retrofitting all US cars with breathalyzers is burdensome. If they already had such equipment, I'd 100% be in favor of gating starting the car on their use, especially if it was a completely passive, foolproof system. You're right on the number of deaths to drunk driving in the US; it's atrocious.

This wouldn't even be the first time in US history a policy change to improve safety and corresponding enforcement occurred; seatbelt laws exist, and enforcement of such laws allows the police to glance in your window to see if you're wearing one and fine you if you aren't. We aren't too far away from cars refusing to be driven if the seatbelt is unclipped.

To swing us back around from this analogous hypothetical to the UK's situation: the cameras are already sunk cost and the question on the table is to improve their utility via software. We aren't talking installing breathalyzers; we're talking about upgrading the existing breathalyzers' firmware to work better at the task they were intended for, so to speak. We can argue whether the software will technically work (separate question, and I'm not in favor of installing something that just doesn't work), but if it does work I'm in favor of passive mass surveillance of public spaces to find persons of interest. It bothers basically nobody and greatly improves the utility of a system that is already installed for the purpose of aiding in stopping crime.

The problem is that neither the government or police are really accountable for abuse of these powers.

As I’ve mentioned elsewhere, the police have admitted previously they’ve abused surveillance powers to blacklist innocent people from construction jobs simply for being members of trade unions. [1]

No body went to prison for that or lost their job and no governments were brought down because of it.

A few companies paid compensation but some are still fighting it in court.

Until we can point to examples members of the police or security service being in someway sanctioned for privacy abuses I don’t think anyone can say that the system is working.

[1] https://www.theguardian.com/uk-news/2019/mar/06/trade-unioni...

Sibling to this thread, it was noted that HUDOC has oversight and can fine a government for human rights violations.

https://www.eff.org/deeplinks/2018/09/uk-surveillance-regime...

Leaving aside any fallout from Brexit, there’s a world of difference between “can” and “has”.

Until safeguards are proven to work and we have examples where they actually haven’t been applied then they effectively don’t exist.

""" Holds, by six votes to one,

(a) that the respondent State is to pay the applicants, within three months from the date on which the judgment becomes final in accordance with Article 44 § 2 of the Convention, the following amounts, to be converted into the currency of the respondent State at the rate applicable at the date of settlement:

(i) to the applicants in the first of the joined cases: EUR 150,000 (one hundred and fifty thousand euros), plus any tax that may be chargeable to the applicants, in respect of costs and expenses;

(ii) to the applicants in the second of the joined cases: EUR 35,000 (thirty-five thousand euros), plus any tax that may be chargeable to the applicants, in respect of costs and expenses; """

https://hudoc.echr.coe.int/eng#_Toc524359882

One can debate how effective a EUR 185,000 fine is in restraining the UK, but it strongly indicates the behavior was unacceptable and will be considered unacceptable moving forward. Seems safeguards work.

> Imagine an organized government subversion cell that is willing to harm civilians to further its objectives and is skilled at hiding in public (one doesn't have to try very hard given the nature of asymmetric warfare in the 21st century).

That cell harming people you want to imagine would be some other government agency acting as part of a covert operation after a long time of planning and preparation, with plenty of spies deep in the government. And they don't actually harm people all that much, not the point, they do just a few acts of terror with a few deaths for war propaganda. But once they are caught on surveillance it's already too late, it means they are ready to approach the head of police and start using serious military weapons, and plenty of people are already on the wrong side of war propaganda.

> where sufficiently-organized individuals use modern technology to harm innocent people and government cannot stop them

Such things don't really happen anymore, governments are too authoritarian already and really like their monopoly on harming people. The only way sufficiently-organized people can coexist with the government is for them to subvert law enforcement or to originate from law enforcement. Governments can deal with either case by replacing everyone or by letting them have their corruption in exchange for being loyal to the government.

Basically surveillance is not the tool for anything you talk about, it's role is the exact opposite: to keep people from disobeying the government, the ruling class, the whole systems, no matter how much harm they do to the people.

The bad guys are the citizens who work to maintain their own ignorance and do nothing to maintain and uphold the rights they enjoy; who don't demand freedom of press; or understand the consequences of these policies, policies eroding their rights to congress and converse freely without fear of suppression.

I don't judge the government for exploring facial recognition technology - there are motivating utilitarian arguments and they have the tools right in front of them. They are doing /something/, even if I think it's deeply wrong. But I do judge those who stick their head in the sand, or sit on their laurels, and that includes myself. We also have motivating arguments and the tools to express them.

I don't want to be part of generation of citizens whose children will say they learned nothing from history - beside the Nuremberg defense.

> The bad guys are the citizens who work to maintain their own ignorance and do nothing to maintain and uphold the rights they enjoy

>But I do judge those who stick their head in the sand, or sit on their laurels, and that includes myself

>I don't want to be part of generation of citizens whose children will say they learned nothing from history

Well you better start funneling funds to the frontlines because talk is cheap.Sounds like you have the technical know-how to do it without ending up on a list. If you were to end up on a list as a result of such contributions, at least it would be a list of the 'good guys' from your own assesment.

No one is going to do it for you so ...

https://tails.boum.org/donate/?r=sidebar

https://supporters.eff.org/donate/join-eff-4

https://donate.torproject.org/

https://riseuplabs.org/

https://donate.mozilla.org

You might even get a cool t-shirt :)

We (in the UK) have an existing system to control this - the warrant system. For the police to search your house, they need to go to a magistrate and provide the reasonable grounds. I would suggest this system works fairly well - I'm sure there are cases where it falls down, but generally the judiciary fairly polices where it is considered reasonable.

I'm in favour of basically every form of government snooping on an individual - whether that's watching your internet activity, tracking your car through ANPR cameras or spotting you with facial recognition - as long as you need to get a warrant to track someone.

The issue seems to be that the police can adopt (some of) these powers arbitrarily and with no due process (and in the past because intelligence agencies operate by a different set of rules). It would also provide an effective limit to the volume of people being tracked in this way, due to the overhead of acquiring a warrant.

Of course, there also needs to be a system to provide oversight and check that only legally authorised individuals are being tracked.

It's not a crazy view but I would add a few things to that

1) The government should not be able compel corporations to add 'features' to help them in this

2) We are talking about an actual warrant, where someone physically went in front of a physical judge int the same location to ask for it, and

3) There is no allowance for speculative but "safe" collection of data they might want under a warrant later - you can only access data occurring after you have the warrant, or naturally occurring records (i.e. no slurping up everyone's cell phone connections/conversations "just in case")

Broad based surveillance of the population by the government is just a bad idea. Cf much of human history on abuse of power.

Number (2) is to make sure this is an expensive process.

I would add that incidental collection is still a problem -- and should the police be able to know the identity of every person in the area of a suspect? What about journalists? John Doe warrants?

If should really be specifically enumerated with no slack. As in, surveillance footage keep for maximum of 1 hour (on line detection only, not historical movement search); only warranted faces will be produced in report, others people blurred.

Also, having unopposed, indefinite warrant applications is bound to be troublesome. Who is checking the validity of police statements? No one. If the police want the surveillance warrant to remain secret indefinitely there should be increasing levels of review. Likewise journalists, politicians, lawyers, medical data, etc.

The UK is one of the most surveilled countries in the world. Growing up in the 90s I began to see cameras appear in public spaces. By the time I emigrated to Canada you could basically be tracked from town to city, cameras on the street, on public transit, and on the motorways.

In Canada I rarely see any evidence of surveillance outside of the cities. Public transit being the exception.

I'm probably slightly younger than you, and I barely notice the CCTV cameras anymore.

Weirdly, the one time I genuinely felt uncomfortable was last week. I saw a traffic camera* swivel across a junction. Actually seeing a camera move in such a way that it was clearly being operated by a human was genuinely unnerving. I took a wrong turn so it wouldn't see me, for no real reason.

*One of these things: https://d1ix0byejyn2u7.cloudfront.net/drive/images/uploads/p...

(comment deleted)
There is something about seeing any kind of surveillance camera move that unnerves me.

At least here in Northern Ireland the traffic cameras can be viewed by the public with either a live video feed or regularly updating images.

> I barely notice the CCTV cameras anymore

Maybe because, as most UK citizens (at least the ones I've spoke to about this), you know that most non traffic cameras are not under police control, but under the control of the establishment that installed them, bars, pubs, stadiums, office buildings etc and the police has to provide a good enough reason, via a warrant, to actually see the footage.

There is also the matter of logistics, to connect every camera in say, London, to a central location to process the data, you would need a crapload of infra that doesn't exists at the moment. Not to mention how much processing power you would need to use this tech for every camera.

I believe the most vocal voices to sort out the false positives will be the police themselves. Deploying the stormtroopers for false positives, a few times per day, is not going to sit well with them.

Interestingly, quality of life doesn't seem to have plummeted in the era of a digitally-mediated panopticon as some sci-fi scenarios predicted.
Because it depends on human operators using intuition to look for suspicious activity. When the machine flags you as a terrorist you will be sent through the system as a CYA precaution.

I knew a Mexican citizen who was a legal green card holder in the US. Unfortunately his name was a match to a fugitive on the most wanted list. Every US entry involved an hour long interview to confirm his identity.

This is what machine vision will to to everyone with its false positives.

> This is what machine vision will do to everyone with its false positives.

That's definitely an impossible scenario, because it's not sustainable to interrogate every entrant into the country for an hour. So it's likelier that the ML algorithm will acquire bias to tone down the likelihood of false positive (at the tradeoff of occasional false negatives).

You misread the comment. Not every entrant. Every entry by any person whoose name matches.
Ironically, match-by-name is what US customs has to fall back on because it lacks a facial-recognition system to make it easy to determine if the person in front of them looks nothing like the person of interest, and a well-implemented facial recognition system could decrease the odds your friend is subjected to an hour-long interrogation for having the "wrong name."
I think the changes have happened slowly enough not to be perceived. Yes, it's not as dramatic as sci-fi, but if you presented today to someone living in the 80s I think they'd recognize enough Orwellian crap going on to fear where their future is headed.

On paper the UK is living in some of the most authoritarian times in modern history. Post 9/11 is a completely different world across much of western society, we have less freedom in many respects. It's almost a trope to say we've given up freedom for the illusion of security. Quality of life isn't entirely represented by how many cars you can afford. I feel like we live in constant fear of terrorism, war, and so on. And yet we never actually get close to these perma-wars, they exist entirely in the press and on TV. I mean, how much Orwellian prophecy do you need before you recognize you are worse off than before?

Here's the thing though: is the situation actually worse than things were for people in the '80s, or do we only believe it must be worse because Orwell wrote some fiction about what life would be like in an era of constant public surveillance, and it turns out the reality on-the-ground is quite different?

Orwell assumed a system with a lot of power that was also unaccountable and heavily abused by those in control of it. UK government is neither of those things; they have elections more often than the US does.

I'm personally of the opinion that people take 1984 as prophetic gospel when there are multiple scenarios that include mass ubiquitous surveillance that could come to pass. David Brin looks through several alternatives in "The Transparent Society" that seem to reflect more closely the nature of the modern world.

Assuming the worst-case scenario is as much an anti-pattern for planning as assuming the best-case scenario.

I can't help but relate the 'First they came ...' prose to the above. I don't really have much to say as I feel the erosion of freedom is clearly apparent and 'worse' than before by all definitions.

From "The Transparent Society" Wiki page (I have not read the book)

> Brin argues that a core level of privacy—protecting our most intimate interactions—may be preserved,

The UK govt. has already invaded intimate interactions, to the point that they got hauled into a Human Rights court

https://www.eff.org/deeplinks/2018/09/uk-surveillance-regime...

I mean, how much more are you going to accept before they finally come for you?

> I mean, how much more are you going to accept before they finally come for you?

Your example is the UK government being roped in by HUDOC, so I believe my answer is "Wake me when the UK isn't answerable to HUDOC."

In particular, the example you've cited is unauthorized and unjustified collection of private correspondence (intercepted communications), not collection of images of people in public and comparison of those images against persons of interest (a thing already legal to do by hand via human operators, which the computer algorithm merely speeds up). Private correspondence is private; why should people believe that their public movements should be so?

> Wake me when the UK isn't answerable to HUDOC

This is basically the 'finally they came for me' argument I was making. You will sleep in until it's too late is what you are suggesting.

> the example you've cited is unauthorized and unjustified collection of private correspondence

This was my point contradicting "The Transparent Society"'s optimistic POV.

I give up.

I don't take Transparent Society's read as optimistic; it highlights dangers but treats those dangers as mitigable if we pay attention.

For example, HUDOC acts as oversight for the risk of overreach of the UK's police apparatus. Given that such oversight exists, the addition of facial recognition to do what the UK camera network is designed to do already (but faster and with programmable parameters) is probably acceptable. The benefits are clear and the risk mitigated by oversight.

This is why facerec is so important -- it, unlike any other technology yet developed, can find abducted people. One suspects there's a silent majority applauding these facerec companies. That majority is defined as anyone who has been a victim of crime.
The near ubiquitous camera coverage in city centers has done nothing at all to decrease the amount of late-night assaults and robberies. More foot patrols when the lager palaces turn out would help, but we can't have that, it would make sense.
also it means giving money to humans to do the policing which is a Bad Thing(tm) according to Tories
That is because the cameras haven't been connected to facerec AIs until now. Now they will be.
"Yup, you were cut up by the same guy that did that acid throwing two weeks back. Our AI says so. We'll give you a heads up if we ever catch him."
Most crimes are against the state, and have gone unpunished until now.
People adapt. Criminals wear helmets.
You're assuming this technology actually works, when in practice it engages in more racial profiling than the also-racist human police force.
Cloud companies definitely will be interested in continue monetizing this. Would be interested to see if Google will finally do it after missing all these juicy government contracts
In a couple of years:

> London to deploy live facial recognition to find unwanted faces in a crowd

One wonders if the populists will offer this a panacea for fear of illegal immigrants.

If they sell it like that I could see the tabloids foaming at the mouth with glee. :(

They deployed something like this in Buenos Aires a few months ago (or maybe a year?). It worked mostly in Subway stations, where there is a lot of people every day passing by.

Although it catched a few fugitives - it triggered a lot of horror stories. A lot of false positives - which in an ideal world wouldn't be that bad. A police man would stop you, asking you a few questions and once he confirms you are not the guy they are looking for, they would leave you free. But it wasn't like this - even when the police man recognized you weren't the one looking for, they have to take you with them to the comisary, and spend a few hours here until a judge can set you free. Worst of all, there are a few stories of people being false recognized on a Sunday afternoon, and had to spend in the comisary until Monday when the Judge could go. Very very sad.

This is deeply disturbing.

The current government recently placed Greenpeace, Extinction Rebellion and cycling advocacy group Critical Mass on a terrorist watch list (even though it admits they aren't terrorist groups). [1]

Previously we've had a situation where a cartel of 14 large construction firms operated a blacklist of workers with political opinions they disagreed with. Some were on the list merely for being members of trade unions.

This blacklist was compiled illegally with the help of UK police's Special Demonstration Squad. [2]

There seems to be a feeling that this sort of technology won't be misused but you can already see the signs that is exactly what will happen.

[1] https://www.independent.co.uk/news/uk/home-news/critical-mas...

[2] https://www.theguardian.com/uk-news/2015/feb/27/on-the-black...

> The current government recently placed Greenpeace, Extinction Rebellion and cycling advocacy group Critical Mass on a terrorist watch list (even though it admits they aren't terrorist groups).

No they didn’t. They placed their symbols alongside many others in a document which aids police forces in identifying both left and right wing groups of various persuasions [1].

Would you rather the police were ignorant of these groups and their associated symbols, such that they mistook peaceful protestors for terrorists?

[1] https://rebellion.earth/wp/wp-content/uploads/2020/01/Counte...

That document makes no mention about which groups are peaceful protestors and which are terrorists. So no one working solely from that list could make that judgement.

If it really was just a list of symbols for groups that might hold a peaceful but potentially disruptive protest then I would expect to see the Countryside Alliance or London Taxis Drivers Association on there... but they are not present.

Policing, by and large, relies on a high degree of consent from the population - you need people who are willing to provide evidence, report crimes, etc. Leaving civil liberties issues and effectiveness in a direct sense aside, doing things like this which don't appear to have public support will result in the loss of a chunk of that consent and make policing harder. This feels a bit short sighted to me.
I wonder how much of this data is being used by politically motivated operators to surveillance their opponents and other inconvenient people. In the US there are a lot of partisans who will do anything that benefits their party (or their own wallet) and will happily pass the data on. I bet a lot of this is already going on with NSA and other agencies’ data without nobody ever knowing.

Pretty scary future we are walking into and it seems there is no way to stop it.

I have a doppelganger very close to my same age. We would often meet out and about since we were in similar circles. People would always remark about our similar looks. We would find out girls had sometimes dated both of us. It was fun and quirky when we were young. But as an adult, I can't help but think of it every time topics of live facial recognition come up.
1984 is not an instruction manual.
I wonder how well this works if people start wearing particulate/surgical face masks. (Wearing masks in public is technically illegal.)

Also wondering if you can jam it by holding up printouts of people in front of cameras.

Gee, just tonight I finished The Capture (2019), which features..London police deploying live facial recognition to find wanted faces in crowds, and numberplates on roads, and the secret service (and a protest group) faking CCTV footage with deep fakes to put anyone they want in prison. I guess I thought the London facial recognition was already a thing. Very good series, by the way.
China, China, China, China, China, OH UK too?
(comment deleted)
Privacy essentials 2020:

1. Use a VPN when connecting to public WiFi

2. DuckDuckGo for web searches

3. Use cash where possible

4. Billy Porter hat