One of my favorite professors would do something like this manually. He would know everyone's names, call on people he thought were slacking or having trouble, and keep everyone alert and engaged and learning because you knew you were being, well, watched.
This seems like it achieves something similar, just automatic. Yet it makes me uneasy. Why? Are human brains unable to cope with the benefits of technology if there's not a consciousness doing it? Or is there something inherently evil about this? Or could it be great if used ethically with ethical rules imposed on it (like, say, the data is never stored for more than an hour).
Scalability. What harm could one professor do by watching who's paying attention? Not all that much. With tech, one software update and hey, now you have a live time series of who's paying attention when- maybe scale attendance scores based on it. Oh, certain kinds of folks aren't recognized as well by the algorithm? Oh, turns out students are being trained to manipulate their facial expressions to seem attentive? Oh well.
Ooh, hey, we have this data on where everybody sits, too- that's handy, since it gives you more information about the social graph of the student body. Reliably sitting next to a Bad Person? Maybe you need some investigating.
And if a professor is caught mentioning subversive information in a lecture? Who was paying attention more than usual during that part? They seem interested, don't they? Might need to give them a talking to, too.
Here's the point. It won't be too long before these children graduate to the job market carrying along some sort of summary score about how good or lazy or naughty they were in school.
When you're at school age, school isn't just "work time", it's most of your social life. There's a big difference to just letting someone watch you work.
Fundamentally it's a violation of trust and consent.
One teacher managing their own classroom allows them to do it on their own terms, personalize their discipline/teaching style to fit the needs of the class, and form a 2-way relationship with the students. Everyone is more or less on the same page and understanding of how and why they're being disciplined, if it comes to that.
Cameras scanning everyone's face once a second and sending them off to who-knows-where for who-knows-what isn't just uncomfortable because it's a 1-way transfer of information with seemingly nothing to offer the students except paranoia, it's applying 19th/20th-century prison surveillance tactics to the classroom. A modern-day panopticon.
I'm not sure what group started it, but there was a political activist group that started destroying CCTV and speed cameras in the UK and U.S. respectively...their main goal was to make it too costly to continually have to replace the damaged cameras...I'm not sure how it went but when I read about I thought it was a great idea...especially since voting won't do much to stop the installation of these surveillance tools...
Of course, I question reasoning behind clients use of our technology, and to date the purposes are universally issues of public safety: Universities, public school districts, sports stadiums, shopping malls, and so on are all highly concerned with public safety and that is the main driver of the adoption of FR.
Keep in mind, FR is not rocket science. (Hell, rocket science is not "rocket science".) It is going to continue to be developed, it is going to be ubiquitous, and the ethics of those in the industry largely define the ethics of the industry itself. If people of good conscious are not in the industry, no ethical concerns will be raised when ethically debatable clients and applications arise. I am part of the good ethics contingent.
I am totally fine with the professor doing it. Kinda like I'm okay with cops patrolling for speeders, but I have a distaste for making it automated with cameras and computers. Using a human enforcer at least evens up the odds, but scaling with cheap sensors and CPUs suggests we could build a world where we actually forbid people from making a choice we disagree with. That is such a fundamental freedom, I think the consequences would be devastating.
Instead of figuring out why children sitting still in a classroom all day doesn't work, we use technology as a threat to force them to pay attention at all times.
A primary purpose of school is to learn and practice social norms and encourage conformity thereto. Taken from that lens, it is perhaps unsurprising that in China, a society moving towards an AI-surveilled and -managed future, such technology would be implemented in schooling as well; a graduating workforce already accustomed to wholesale observation will not complain about it elsewhere in their daily lives.
Whatever you want to call them, the basics/classics/fundamentals are all just primers to get your mind firing on all cylinders. Having access to all the knowledge in the world is useless if you don't know what questions to ask or what direction to take. And since everyone doesn't learn in the same manner, exposure to a wide variety of subject matter eventually works well enough for most people.
Particularly to my Western values of freedom and privacy and individuality. Much of what China does seems like that. Horrible and anti-human.
What concerns me most however, is that this (and similar Orwellian schemes) might actually be truly effective, and our fixation in the West with individual liberties might not be competitive in the long run.
There are lots of effective things - enforce a healthy diet, monitor everyone continuously, reduce the population to improve the environment, put criminals to death.
The goal of a society should not be "to be effective, efficient, and cost-justified". Other values must be considered.
I believe effective culture in this context should be understood as one able to produce and sustain larger number of people, and spread it's influence to more societies.
Reducing population for instance is clearly not effective, because society that does it will be replaced with the neighbour that doesn't.
Putting criminals to death doesn't help much as well, and can harm when an innocent person is killed by mistake.
In this sense western values have been very effective so far, but as technology changes, they may need to change.
Personally i don't think that privacy by itself is a good thing, if there was a device allowing anyone to see video of any place at any time (eliminating all privacy) people would be better off.
Things get bad only if a small group that already had control over the society gets exclusive access to the information, further reducing the power of other groups. This is bad, because it reduces the complexity of the society and doesn't allow to explore more paths (which in the long run have always failed so far).
But if the powerful group is smart enough it may still make society more effective, by reducing the crime and vandalism, by helping to develop better school programs etc.
So if western societies want to remain effective the answer can't be simply 'no cameras' it should be cameras plus better mechanisms for distributing the power of the state between more groups.
I disagree. I think that "no cameras" by itself is a useful value, because surveillance makes humans behave differently.[0] People have higher levels of stress, are more likely to conform, and probably more likely to engage in civic life. Perhaps the cameras themselves will make surveillance states less competitive?
I think that, ultimately, your last point will be true.
Many things about liberty in places like the UK seem to be treated as some kind of dogma rather than something that we can use to improve and enjoy life more. So people spend a lot of time arguing about them, but don't seem to do much with them (other than argue, I guess).
The attitudes towards the type of things in this article are also much more positive from people I talk to in China. Not just random samples online, people that I know and trust. The social credit system for example, people are generally a bit uneasy about it (given the past of their country) but not against it, the practical impact that it can have to them seems like it's worth it.
Technology in general seems to be met with much less cynicism and doubt there than the UK.
So personally I think that I agree with the notion that these kinds of systems truly have the potential to be effective, and that somewhere like the UK is going to be incapable of harnessing them in the way that somewhere like China is. And as this becomes increasingly apparent the reaction that other countries will have will be interesting and perhaps scary. Will america become increasingly confrontational? Try to instigate something in Taiwan? Idk.
2025 is the year that I've chosen to wait until before I make a judgement as to how messed up things really are.
I fear it's something worse than effective. At some point somebody will come with a way to make it effective at the short term, with disastrous long term consequences.
Disastrous long-term consequences are scarier as long as they have some short term benefits. At least with long-term success you have something positive at the end.
Humans, particularly politicians and young people, tend to have a high time preference. That is, they put more emphasis on the here and now over future deferred rewards and consequences. This means that if a strategy has good short term rewards, then people will often ignore the long-term effects. Even if you can show that the long-term effects are negative, they are likely to ignore it. Eg moving towards socialism. We have plenty of evidenced from past attempts that socialism is likely not going to work, but people still like advocating for such policies, because they have a positive impact in the short term.
All of this surveillance with cameras from china articles has me highly skeptical. Can CCTV quality cameras really be used for on the fly person recognition, including determining if an identified student is 'focused' or 'dozing off'?
I keep thinking that this type of surveillance from China is all smoke and mirrors, to make it seem like they have more of a lock down than they actually do. Even if an automated person recognition/mood judgement system was 99% accurate for a public CCTV camera, that's 10 false positives per 1000 people, and consider how many people it would see a day. But I don't work in image/video processing, so idk. But I think it's harder than identifying crosswalks, fire hydrants, and storefronts.
In a classroom, it only needs to recognize 30 people. That seems doable.
Car makers also claim to work on detecting dozing off of drivers. What about it should be impossible? Detecting if eyes are open or shut sounds doable, unless everybody is wearing sunglasses.
All it needs to be is smoke and mirrors, with a bit of human oversight to give it credibility. Even if it only has a 50% accuracy, an administrator can review the flags and take action, discouraging that activity for everyone.
When I was teaching last time in China, all classrooms in the university I was a guest at had cameras installed in the back, facing the lecturer. The last time I was there, only some had them.
What are these for, I asked my host, wondering if it had to do with making sure courses were in line with party policy and so on.
"Mainly, it is to make sure you start and end courses on time. You need to start exactly at the strike of the bell and stay until the next bell rings."
What happens when you just turn out to be done with the material for the day, I asked, or you see the material was heavy and students have lost focus. In those cases I often end a bit earlier with my students.
"We can't leave. In most cases we just gossip a bit with the students until class is done."
Students didn't seem to bother, back then, most Chinese don't make a taboo of sleeping so some were close to drifting off. They're overworked and quiet, so I didn't mind that much either.
Is there a whole team checking all courses simultaneously, I inquired.
"No, sometimes no one will be watching. They're 'government officers' [officer is a common English translation used for anything government and administrative]... I don't think they care too much either. Their job is to make sure we do our job, you know."
With the advances in AI, the dream becomes a possibility. Watching a whole society. Even if you're not sure for what or it 'doesn't matter' what you're actually doing, the fact that something is always watching seems to be powerful enough on its own. It's for a large part about inspiring the idea of just behaving and executing your given steps, and you'll be fine.
Meta-commentary: Sixth Tone is a propaganda outlet controlled by the Chinese government. So why are they running with this story?
- The central government's AI strategy is never directly criticized. Instead you get quotes like
She agrees that the intention of “intelligent education” is positive. As the NGAIDP guidelines suggest, the purpose of the initiative is to assist teachers in developing customized teaching methods and study plans for every student.
- On ubiquitous surveillance cameras, the example given is finding criminals in a crowd, not rounding up dissidents.
- The parts that are criticized are more about the implementation than the intention behind it. The potential for misuse isn't even mentioned, the focus is on potential lack of accuracy instead.
- Responsibility is shifted to the schools and companies, despite following the government's plan:
The plan’s “intelligent education” section describes in detail how China’s government hopes to use AI to boost the country’s education system. Zhang reads me one paragraph from the guidelines without stammering. “So detailed. It’s like they wrote it with a [facial recognition] product right in front of them,” he says.
What I think this all means is that Sixth Tone is trying to be a source for high-quality journalism on social issues in China while conveniently avoiding topics the government would like to remain unmentioned.
> Meta-commentary: Sixth Tone is a propaganda outlet controlled by the Chinese government. So why are they running with this story?
No they’re not. They’re journalists living and working in China who publish pieces they don’t anticipate will get them shut down. They’re not going to cover the concentration camps in Xinjiang but it’s not an actual propaganda outlet like Xinhua.
> What I think this all means is that Sixth Tone is trying to be a source for high-quality journalism on social issues in China while conveniently avoiding topics the government would like to remain unmentioned.
It’s like they expect their readers to be intelligent people who can read between the lines.
>It’s like they expect their readers to be intelligent people who can read between the lines.
To put it bluntly: this is what state censorship does to news. Even if they wanted to do better, they can't, because they'll get shut down or punished.
Sixth Tone isn't directed at the Chinese market though, getting blocked by the government shouldn't be a problem for them, as SCMP proves. The difference is that SCMP is owned by Alibaba, while Sixth Tone is part of the Shanghai United Media Group, who are controlled by the party committee of Shanghai. The journalists might not think of their work as anything bad (宣传 doesn't have a negative connotation in Chinese, after all) and I don't think they'll publish outright lies, but they wouldn't exist if not to influence public opinion by countering the usually negative reporting on China with a more positive spin.
That doesn't mean I advise against reading them (I did say that they publish high-quality articles, after all) but you need to be aware that they're not going to tell the whole truth.
Sixth Tone isn't a propaganda outlet though, every media source in China has to get their news approved before they can publish it. The stuff in there is there because it had to be to get published.
And I quote: "For instance, in November 2018, Chinese authorities wrongly accused entrepreneur Dong Mingzhu of jaywalking after a streetside camera identified her face in an ad on the side of a bus." - this is clearly not the work of a propagandist.
Whilst the facial recognition aspect of this and the Chinese tie-ins with total monitoring of its populace are one aspect, in schools (at least in Australia, and I assume many others) large amounts of centralised data on behaviour, performance, etc are stored about students by their teachers already.
One of the comments here makes it sound all so personal, as if that information never leaves the classroom and is only the domain of the teacher, but it isn't - it's shared with other teachers, heads of year, deputies and principals, school psychologists, government departments, and to some extent (usually in the aggregate) with parents.
Throw some automation in the mix for scale (along with all the issues with correct facial identification - which as teachers we're not immune to either!) and we have the same thing.
Freder looked across the city at the building known to the world as the "New Tower of Babel."
In the brain-pan of this New Tower of Babel lived the man who was himself the Brain of Metropolis.
As long as the man over there, who was nothing but work, despising sleep, eating and drinking mechanically, pressed his fingers on the blue metal plate, which apart from himself, no man had ever touched, so long would the voice of the machine-city of Metropolis roar for food, for food, for food...
She wanted living men for food.
Then the living food came pushing along in masses. Along the street it came, along its own street which never crossed with other people's streets. It rolled on, a broad, an endless stream. The stream was twelve files deep. They walked in even step. Men, men, men--all in the same uniform, from throat to ankle in dark blue linen, bare feet in the same hard shoes, hair tightly pressed down by the same black caps.
And they all had the same faces. And they all appeared to be of the same age. They held themselves straightened up, but not straight. They did not raise their heads, they pushed them forward. They planted their feet forward, but they did not walk. The open gates of the New Tower of Babel, the machine center of Metropolis, gulped the masses down.
40 comments
[ 4.8 ms ] story [ 98.4 ms ] threadThis seems like it achieves something similar, just automatic. Yet it makes me uneasy. Why? Are human brains unable to cope with the benefits of technology if there's not a consciousness doing it? Or is there something inherently evil about this? Or could it be great if used ethically with ethical rules imposed on it (like, say, the data is never stored for more than an hour).
Ooh, hey, we have this data on where everybody sits, too- that's handy, since it gives you more information about the social graph of the student body. Reliably sitting next to a Bad Person? Maybe you need some investigating.
And if a professor is caught mentioning subversive information in a lecture? Who was paying attention more than usual during that part? They seem interested, don't they? Might need to give them a talking to, too.
Apparently some people have a stranger watch them work as a productivity hack [1].
Of course, consent makes all the difference.
[1] https://news.ycombinator.com/item?id=19591227
One teacher managing their own classroom allows them to do it on their own terms, personalize their discipline/teaching style to fit the needs of the class, and form a 2-way relationship with the students. Everyone is more or less on the same page and understanding of how and why they're being disciplined, if it comes to that.
Cameras scanning everyone's face once a second and sending them off to who-knows-where for who-knows-what isn't just uncomfortable because it's a 1-way transfer of information with seemingly nothing to offer the students except paranoia, it's applying 19th/20th-century prison surveillance tactics to the classroom. A modern-day panopticon.
Do you ever question your line of work and what good can come from the facial recognition technology you help develop?
Keep in mind, FR is not rocket science. (Hell, rocket science is not "rocket science".) It is going to continue to be developed, it is going to be ubiquitous, and the ethics of those in the industry largely define the ethics of the industry itself. If people of good conscious are not in the industry, no ethical concerns will be raised when ethically debatable clients and applications arise. I am part of the good ethics contingent.
What's the point of going to school if all the world knowledge is in your pocket?
To learn how to apply said knowledge?
Particularly to my Western values of freedom and privacy and individuality. Much of what China does seems like that. Horrible and anti-human.
What concerns me most however, is that this (and similar Orwellian schemes) might actually be truly effective, and our fixation in the West with individual liberties might not be competitive in the long run.
The goal of a society should not be "to be effective, efficient, and cost-justified". Other values must be considered.
Reducing population for instance is clearly not effective, because society that does it will be replaced with the neighbour that doesn't.
Putting criminals to death doesn't help much as well, and can harm when an innocent person is killed by mistake.
In this sense western values have been very effective so far, but as technology changes, they may need to change.
Personally i don't think that privacy by itself is a good thing, if there was a device allowing anyone to see video of any place at any time (eliminating all privacy) people would be better off.
Things get bad only if a small group that already had control over the society gets exclusive access to the information, further reducing the power of other groups. This is bad, because it reduces the complexity of the society and doesn't allow to explore more paths (which in the long run have always failed so far). But if the powerful group is smart enough it may still make society more effective, by reducing the crime and vandalism, by helping to develop better school programs etc.
So if western societies want to remain effective the answer can't be simply 'no cameras' it should be cameras plus better mechanisms for distributing the power of the state between more groups.
[0] https://www.psychologytoday.com/us/articles/201409/all-eyes-...
The attitudes towards the type of things in this article are also much more positive from people I talk to in China. Not just random samples online, people that I know and trust. The social credit system for example, people are generally a bit uneasy about it (given the past of their country) but not against it, the practical impact that it can have to them seems like it's worth it.
Technology in general seems to be met with much less cynicism and doubt there than the UK.
So personally I think that I agree with the notion that these kinds of systems truly have the potential to be effective, and that somewhere like the UK is going to be incapable of harnessing them in the way that somewhere like China is. And as this becomes increasingly apparent the reaction that other countries will have will be interesting and perhaps scary. Will america become increasingly confrontational? Try to instigate something in Taiwan? Idk.
2025 is the year that I've chosen to wait until before I make a judgement as to how messed up things really are.
Humans, particularly politicians and young people, tend to have a high time preference. That is, they put more emphasis on the here and now over future deferred rewards and consequences. This means that if a strategy has good short term rewards, then people will often ignore the long-term effects. Even if you can show that the long-term effects are negative, they are likely to ignore it. Eg moving towards socialism. We have plenty of evidenced from past attempts that socialism is likely not going to work, but people still like advocating for such policies, because they have a positive impact in the short term.
I keep thinking that this type of surveillance from China is all smoke and mirrors, to make it seem like they have more of a lock down than they actually do. Even if an automated person recognition/mood judgement system was 99% accurate for a public CCTV camera, that's 10 false positives per 1000 people, and consider how many people it would see a day. But I don't work in image/video processing, so idk. But I think it's harder than identifying crosswalks, fire hydrants, and storefronts.
Car makers also claim to work on detecting dozing off of drivers. What about it should be impossible? Detecting if eyes are open or shut sounds doable, unless everybody is wearing sunglasses.
The same applies for identifying dissidents.
What are these for, I asked my host, wondering if it had to do with making sure courses were in line with party policy and so on.
"Mainly, it is to make sure you start and end courses on time. You need to start exactly at the strike of the bell and stay until the next bell rings."
What happens when you just turn out to be done with the material for the day, I asked, or you see the material was heavy and students have lost focus. In those cases I often end a bit earlier with my students.
"We can't leave. In most cases we just gossip a bit with the students until class is done."
Students didn't seem to bother, back then, most Chinese don't make a taboo of sleeping so some were close to drifting off. They're overworked and quiet, so I didn't mind that much either.
Is there a whole team checking all courses simultaneously, I inquired.
"No, sometimes no one will be watching. They're 'government officers' [officer is a common English translation used for anything government and administrative]... I don't think they care too much either. Their job is to make sure we do our job, you know."
With the advances in AI, the dream becomes a possibility. Watching a whole society. Even if you're not sure for what or it 'doesn't matter' what you're actually doing, the fact that something is always watching seems to be powerful enough on its own. It's for a large part about inspiring the idea of just behaving and executing your given steps, and you'll be fine.
- The central government's AI strategy is never directly criticized. Instead you get quotes like
She agrees that the intention of “intelligent education” is positive. As the NGAIDP guidelines suggest, the purpose of the initiative is to assist teachers in developing customized teaching methods and study plans for every student.
- On ubiquitous surveillance cameras, the example given is finding criminals in a crowd, not rounding up dissidents.
- The parts that are criticized are more about the implementation than the intention behind it. The potential for misuse isn't even mentioned, the focus is on potential lack of accuracy instead.
- Responsibility is shifted to the schools and companies, despite following the government's plan:
The plan’s “intelligent education” section describes in detail how China’s government hopes to use AI to boost the country’s education system. Zhang reads me one paragraph from the guidelines without stammering. “So detailed. It’s like they wrote it with a [facial recognition] product right in front of them,” he says.
What I think this all means is that Sixth Tone is trying to be a source for high-quality journalism on social issues in China while conveniently avoiding topics the government would like to remain unmentioned.
No they’re not. They’re journalists living and working in China who publish pieces they don’t anticipate will get them shut down. They’re not going to cover the concentration camps in Xinjiang but it’s not an actual propaganda outlet like Xinhua.
> What I think this all means is that Sixth Tone is trying to be a source for high-quality journalism on social issues in China while conveniently avoiding topics the government would like to remain unmentioned.
It’s like they expect their readers to be intelligent people who can read between the lines.
To put it bluntly: this is what state censorship does to news. Even if they wanted to do better, they can't, because they'll get shut down or punished.
That doesn't mean I advise against reading them (I did say that they publish high-quality articles, after all) but you need to be aware that they're not going to tell the whole truth.
And I quote: "For instance, in November 2018, Chinese authorities wrongly accused entrepreneur Dong Mingzhu of jaywalking after a streetside camera identified her face in an ad on the side of a bus." - this is clearly not the work of a propagandist.
One of the comments here makes it sound all so personal, as if that information never leaves the classroom and is only the domain of the teacher, but it isn't - it's shared with other teachers, heads of year, deputies and principals, school psychologists, government departments, and to some extent (usually in the aggregate) with parents.
Throw some automation in the mix for scale (along with all the issues with correct facial identification - which as teachers we're not immune to either!) and we have the same thing.
In the brain-pan of this New Tower of Babel lived the man who was himself the Brain of Metropolis.
As long as the man over there, who was nothing but work, despising sleep, eating and drinking mechanically, pressed his fingers on the blue metal plate, which apart from himself, no man had ever touched, so long would the voice of the machine-city of Metropolis roar for food, for food, for food...
She wanted living men for food.
Then the living food came pushing along in masses. Along the street it came, along its own street which never crossed with other people's streets. It rolled on, a broad, an endless stream. The stream was twelve files deep. They walked in even step. Men, men, men--all in the same uniform, from throat to ankle in dark blue linen, bare feet in the same hard shoes, hair tightly pressed down by the same black caps.
And they all had the same faces. And they all appeared to be of the same age. They held themselves straightened up, but not straight. They did not raise their heads, they pushed them forward. They planted their feet forward, but they did not walk. The open gates of the New Tower of Babel, the machine center of Metropolis, gulped the masses down.
An snippet from Metropolis by Thea Von Harbou