Talking to strangers. On the Use of Emotional AI to Prevent and Combat Online Grooming and Sex Chatting with Minors

At the intersection of emerging AI technologies and the protection of minors against online sexual abuse, fascinating yet worrisome developments unfurl before our eyes. On the one hand, various new forms of online text-based sexual behaviour aimed at minors (online grooming, solicitation of minors, attempted online grooming and online sex chatting) are brought within the scope of criminal law, thus leading to forms of preventive or risk-based criminal law. On the other hand, various (emotional) AI technologies that process and analyse textual/linguistic data are more and more commonly employed as investigative tools to detect or prevent these kinds of (criminal) behaviour or catch online predators in the act. However, not only do these technologies still fall short in many respects and are therefore demonstrably ineffective but their use also poses a potential threat to fundamental rights and online values such as privacy, freedom of expression, confidentiality of communication, online security and safety, especially of the very children they are supposed to protect. This chapter offers a critical review of some of the possible consequences of AI-driven prevention and detectionDetection of online text-based sexual offenses, particularly by the use of police bots and large scale, automated scanning of online speech.