Capturing the Semantics of Internet Memes

Internet memes have emerged as a novel format for expressing ideas on the web. They have evolved from a simple form of entertainment to a sophisticated medium of communication that reflects societal sentiments, values, trends, and occasionally unethical and harmful behavior. Internet memes are multimodal, succinct, relatable, and fluid, making them challenging and inspiring for many stakeholders, including content moderators, marketing strategists, and social science researchers. Interpreting internet memes is also a challenging objective for (neuro-symbolic) AI, requiring knowledge about memes to be collected, curated, enriched, and integrated with multimodal processing models. This chapter provides background on internet memes and discusses key considerations in machines’ interpretation of memes towards developing robust and explainable methods that can estimate meme similarity, detect toxic memes at scale, and reason about the impact of cultural and personal values. We discuss challenges for meme knowledge collection and curation and describe our approach to generating the Internet Meme Knowledge Graph. We describe methods for estimating meme similarity to ground memes from the web, designed to capture content, form, identity, and stance. We review ongoing efforts to develop explainable hate speech detection methods in memes. Finally, we list open challenges and promising future directions for reliable AI reasoning over internet memes.