Discuss HN: How should tech be used to fight child abuse?
Developments in "tech" are without a shadow of a doubt one of the greatest achievements of the last 30 years. When used right, they improve our lives, prolong our lives, save our lives. It seems obvious that there should be an application of technology that helps fight crime, child abuse in particular.
But how should that work? A debate where "techies" just say nay to what seems like a win-win for fighting child abuse is incomplete. The world and the big corporations won't stop for us. We need to either suggest better alternatives we can push in the public debate, or at least come to terms with the change, and think about mitigating its negative impact.
I don't have great suggestions off the top of my head, but turning some ideas around:
- Make device searches warrant-driven. Law Enforcement would be able to submit (perhaps streamlined, electronic) warrant applications, to be reviewed by court, backed up by other evidence. For something like matching photo hashes, the bar could be lower than e.g. for searching someone's house. The flipside would be that if the warrant is found to be inadequately justified (even in retrospect), no data found in it is available for evidence.
- On a similar vein, perhaps Law Enforcement only gets very limited information back from such a warrant. For example, only photos matching hashes are available back to them. This severely reduces the usefulness of this as a general fishing tool.
- Sadly, I think just saying "let's not scan users' devices" is like trying to ban nuclear weapons. The cat is out of the bag, it can be done, so it will be done. But perhaps not? Is there an alternative that people with low technology awareness would see as adequate?
- What non-tech analogies can be made and leveraged in the discussion? A recent HN submission [1] compares the change to mandatory weekly house searches; I don't think this would resonate with the general public. House searches are invasive and uncomfortable, much more so than devices, which we all know have a life of their own. Is it like a partner constantly going through our wallets or briefcase/handbag? Or going through our messages all the time? What about mobile phone data already available to Law Enforcement - like it or not, they can access our (approximate) location, call logs, bank transactions etc. with a similarly low level of scrutiny. To what extent is scanning photos on our devices different?
I hope this can lead to a fruitful discussion on the positive ways technology can be used to fight crime, minimising the impact on innocent people.
[1] https://news.ycombinator.com/item?id=28127828
EDIT: added missing reference
17 comments
[ 340 ms ] story [ 1468 ms ] threadChild abuse is a social problem. Any tech solution is a bandaid that may reduce abuse, but we should be aiming to prevent it.
I don't know what role tech has in that, but it could help to provide better mental health resources for potential offenders and better support/escape options for victims.
With free software, you can verify that your device does not scan you. Or hire someone to verify that. This is what I choose for myself.
Also, you might protect yourself like this, but what about people you message and share data with? Gmail sees the emails of its own users and their conversations.
Injecting sanity into things like Apple’s CSAM scanning matters to everyone.
I see tech in a role to provide information about symptoms of abuse. Tech is sadly too often deployed for surveillance purposes and I think people working in that field give it a very bad name.
On the other hand I wouldn't want my 14 year old kid to be exposed to everything the net has to offer. But I also don't want to put them under surveillance, they will only become compliant idiots that way. I am aware that most mechanisms will be doomed to fail because they will very likely surpass my abilities with anything technical. I have few ideas how to implement this without a great firewall for kids.
Still, would be nice to a compromise between a catholic boarding school for girls and modern abusive teenage social networks.
You have to leave behind some dogmatism prevalent in tech, as some naivete is worth preserving, not only for kids btw.
Putting people under general suspicion is a complete wrong way and while I think chastising Apple isn't too sincere, I don't mind people doing it. Compliance like this should be punished. Although that should also be true for Google and co.
If we want to avoid a dystopian tech slavery, we have to actively provide positive alternatives to “just more invigilation“.
You talk about protecting children from content, eg via firewalls. What about the production of the “content”? Do you think there isn’t a role to play for technology?
> If we want to avoid a dystopian tech slavery, we have to actively provide positive alternatives to “just more invigilation“.
No, you should always block any form of "cooperation" right there. Don't ever provide material so that some moderately talented official can build a profile. Basic lesson of pedagogy is to deny these attempts in the first place. It is what Apple should have done because it probably won't help child abuse anyway and it won't stop with this kind of content.
Always defer them to investments in child care and defect their unproductive ambitions with real means.
The Apple stuff feels for me that is just the begining, observe how contradictory it is
1 the scan happens only if the bad guy is an IDIOT and enables iCloud to backup is porn collection
2 the bad guy is NOT and IDIOT and he applied filters,copping so we need to use Bob fancy alogirthm to catch this alterations.
So either 1 is true then 2 implies that they put the fancy algorithm in because some dude with big ego wanted to add that to his CV and so causing innocent people stress , or 2 is true , then 1 is just a temporary check that will be toggled off at next update by Apple after some concerned citizens will complain.
As competent software developers we should know that when you have a bug you always find the root problem, even if you maybe have to rush a quick dirty fix you should find the root problem so it does not cause other weird issues.
As software developers we also understand that giants do not care for a individual users, they are expandable, we can see from Google , Sony and other giants how you can be flagged and screwed over because some false positiveness or a bad actor abusing the system.
In the Apple case. if the bad guys are idiots then scan the server side, reported the bad guy, then when police commes with a warrant help them to obtain what they need about the bad guy.
The main concern is when the device has software that is not protecting you from bad stuff but is snitching on your activity. Next they will go further and screengrab your screen, spy on your browsing, read your messages, listen to your conversation and what radio/tv or other media device you are listening/watching.
From what I read in comments it seems most abuses are done by a family member. so if such a person abuses a child, takes some pictures , this system will not work at all, it seems to me you need a social solution and not putting some stupid AI in all cameras or other weird AI solutions developers with big ego dream.
They were looking for volunteers sometime back to classify online abuse content and one of the requirement was "This is a tough job and only emotionally resilient people should apply....This is not a job for those on a personal mission, or for those who are easily upset by illegal online content." But now they are looking for paid part-time employees[1] for the same job. It is a tough job. There are other jobs available with them that are not that emotionally taxing.
[0] https://www.iwf.org.uk
[1] https://www.iwf.org.uk/what-we-do/who-we-are/work-us
I'd add that I have served on the District of Columbia Infant Mortality Review Committee. No lie, 9/10 of the time, the recommendations were breastfeeding, parent support, parent coaching, reducing the number of births in a family. It took YEARS before any of the recommendations were implemented.
As to the use of tech, I'd suggest talking with practitioners and policy makers. For example, child advocacy centers (in the US, at least) may have some ideas or suggestions of other ways to help reduce the incidence of child abuse and neglect.
The bar is pretty low already. Anything found by warrant that is found to be defective is thrown out already under the fruit of the poisonous tree theory.
Even if the evidence is thrown out, or if the person is innocent, the process can still ruin people's lives, cost money, etc. I recently had an experience with a trooper letting a dangerous dog charge stand even though he knew it was the wrong charge. That charge carried with it pretrial restrictions. This should be a violation of the 8th amendment's prohibition on unusual punishment. There were a few other issues/rights violation with what he and prosecutors did too. Anyways, nobody cared - they even said they are allowed to leave incorrect charges stand. Unless you spend thousands of dollars on a lawyer, you're screwed. Even if you're right, they will not take you seriously unless you have a lawyer. It generally costs more to defend your innocence than pay the fine for lower level offenses.
Giving the system even more power is just going to lead more innocent people being trampled on. I would guess it's only a matter of time before some parent gets caught in Apple's scanning because they have a naked picture of their baby taking a bath or something.
Can we make a chatbot that is provably local or erased, anonymous, open, and yet able to offer meaningful support or assistance?
Can we offer communication channels with similar reassurances to kids who need them?
As for using technology to fight crime, see how the quality of your local police IT infrastructure looks. There's alot of room for improvement there and its all stuff that "tech" is really good at, paperwork replacement and information access.