Session

Speaking the Language of Hate: What AI Misses About Online Extremist Subcultures

When online extremist subcultures communicate, they rarely rely on the explicit slurs or obvious profanities that standard moderation filters are built to catch. Instead, they construct elaborate "cryptolects", private, rapidly evolving dialects built from altered acronyms, morphological suffixes, and fatalistic jargon designed to enforce in‑group identity while flying under the radar of automated safety systems. And some of that language has now leaked into everyday use. “Looksmaxxing” is practically mainstream but it started as incel jargon long before TikTok picked it up.

When commercial moderation models encounter this language, they stumble. General‑purpose moderation models get tripped up by words with multiple meanings and short tokens that collide, so they miss high‑risk subcultural signals while over‑flagging harmless conversations that use the same terms.

To understand why AI misses the mark, I conducted a deep‑dive investigation into incel subcultures using a dataset of 1.8 million posts from r/Braincels. By building a multi‑label taxonomy of over 400 subcultural terms, I benchmarked off‑the‑shelf commercial moderation APIs against a domain‑adapted HateBERT model trained specifically on subcultural context.

In this talk, we’ll step into the intersection of digital anthropology and applied NLP. We’ll explore how online communities actively engineer language to evade detection, why surface‑level toxicity filters fail in high‑context subcultures, and how context‑aware feature engineering can help AI systems understand what words are actually doing in a conversation rather than just counting bad words.


Why commercial AI moderation tools fail when faced with subcultural "cryptolects" and how digital anthropology and domain-adapted NLP can bridge the gap between pattern matching and true subcultural comprehension.

Lianne Potter

AI Security, Safety & Ethics Leader | Technologist & Digital Anthropologist | Co-Founder, NorthStar Intelligence

Leeds, United Kingdom

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top