Last week Prostasia Foundation participated in an event titled "Preventing Child Online Grooming: Working Together for Maximum Impact" which was organized by the UK Home Office, the WePROTECT Global Alliance, and Microsoft. A surprise proposal at the event came from Hany Farid, professor of computer science at Dartmouth College. Professor Farid is one of the developers of the PhotoDNA technology that Microsoft and its partners use to scan images that are uploaded by users to the web. Professor Farid's suggestion was that this scanning technology should be incorporated into all major web browsers.
This raises several privacy and security concerns. The hash databases used with PhotoDNA, as compiled by organizations such as the Internet Watch Foundation (IWF) and the National Center for Missing and Exploited Children (NCMEC), are meant to contain only unlawful images of minors, but due to the nature of these images, there is no transparency about what images are included. As such, lawful images have previously been wrongly included in these databases, resulting in false positive matches that impacted innocent Internet users.
But more alarming than this is that by moving this hash-scanning technology from the server side (scanning by web companies) to the client side (scanning in your own web browser), you are allowing a "Big Brother" into your home. Exactly as happened with server side scanning, governments and special interest groups will soon insist on expanding it to scan for other types of illicit content, such as copyright-infringing downloads. Since currently copyright owners complain about the game of "whack-a-mole" that they play when copyright files are moved from one server to another, incorporating copyright-scanning technology directly into the web browser could be an irresistable solution. It may also prove irresistable to governments that wish to censor downloads of banned security tools like secure messaging applications.
Perhaps even more concerning still is the prospect that not only our downloads, but also our private messages may soon be being automatically scanned. The main outcome of the Microsoft event was the announcement of a new surveillance technology that is claimed to be able to to automatically flag potential conversations taking place between child groomers and children. Prostasia Foundation will be seeking further details of this technology and requesting access to an early beta for evaluation as part of our planned 2019 Internet Platform Review. We intend to use logs of chats of sexual role play, sourced consensually from the adult participants to those chats, to determine whether the technology is as accurate as claimed at identifying actual cases of attempted grooming.
When we raised some of these concerns with Professor Farid, he dismissed them. But this is a classic illustration of the need for vigilance to ensure that in our understandable and laudable enthusiasm to thwart child sexual abusers, we don't overlook significant civil liberties concerns about the potential for some of these technologies to harm the privacy and security of innocent Internet users. We will be inviting Dr Farid to attend our May 2019 meeting of Internet platforms and experts, to ensure that our concerns are conveyed to the technologists and policymakers who are leading this charge.