TALKING TO STRANGERS: On the use of emotional AI to prevent and combat online grooming and sex chatting with minors

At the intersection of the latest AI technologies and the protection of minors against online sexual abuse, fascinating yet worrisome developments unfurl before our eyes. On the one hand, various new forms of online text-based sexual behaviour aimed at minors (online grooming, solicitation of minors, attempted online grooming and online sex chatting) are brought within the scope of criminal law, thus leading to forms of preventive or risk-based criminal law. On the other hand, various emotional AI technologies that process and analyse textual/linguistic data are more and more commonly employed as investigative tools to detect or prevent these kinds of (criminal) behavior or catch online predators in the act. However, these technologies not only still fall short in many respects and are therefore demonstrably ineffective, their use also poses a potential threat to fundamental rights and online values (such as privacy, freedom of expression, confidentiality of communication, online security and safety), also and especially of the very children they wish to protect. This chapter offers a critical review of some of the possible consequences of AI-driven prevention and detection of online text-based sexual offenses, especially by the use of police bots and large scale, automated scanning of online speech.