Self-written explanations are what let models read dark humor in memes

Not all jokes work the same way. Clean humor runs on wordplay and harmless incongruity; dark humor runs on painful subjects, cultural references and fine contrasts between the image and the caption. In memes this is especially visible: the picture says one thing, the text says another, and the meaning appears where they meet. Until recently there was no good multimodal dataset for dark humor specifically, and models kept confusing irony, sarcasm and plain aggression. The D-Humor team closes that gap: they assembled a dataset of memes and proposed a way to teach models to reason — not just look and read, but also explain what exactly makes a joke dark.

Why this matters

Understanding dark humor is about context, not censorship. Platforms need to tell malicious speech apart from a humorous comment, however harsh it is. Researchers want to see which groups end up as targets most often. And for the people building AI, it is a way to find where a model fails because of cultural differences or ambiguous visual cues.

How the data was collected

The authors collected 4,397 memes from Reddit, keeping both the image and the OCR text pulled from it. Every meme was labeled along three axes:

• whether it is dark humor;
• the target of the joke: gender/sex, mental health, violence/death, race/ethnicity, disability, other;
• intensity: mild, moderate, high.

Labeling was done by three trained annotators with regular cross-checks. The split between dark and non-dark humor is close to even. The most frequent targets are gender/sex and the "other" category, and milder cases dominate the intensity scale. The team makes a point of the ethics: the data is sensitive, access is granted under an agreement, and no personal data is included.

Warning: examples from the dataset may contain offensive content

How the proposed method works

The core idea is to add one more layer on top of the image and the text: a structured explanation that the model writes itself. The authors take a VLM (Qwen-2.5-7B) and ask it to break the meme down briefly across six fields: what is happening in it, where the punchline is hidden, how the narrative is built, what feeling it produces, which dark cues are present, and who the potential target is. Then they run iterative self-improvement: the model steps into the role of the meme's author, criticizes its own explanation and refines it. Three iterations are usually enough to make the explanation fuller and more coherent.

Then comes feature extraction. The OCR text goes through BERT, the explanation through S-BERT, the image through ViT. Where they meet sits the Tri-stream Cross-Reasoning Network (TCRNet): it matches text, image and explanation pairwise, catches the mismatches and fuses everything into a single representation. Those mismatches are often what produces the dark humor in the first place, so catching them is critical.

What the experiments showed

Baseline language models are already decent at recognizing dark humor, but they get weaker on the target and weaker still on intensity. Vision models do worse. Zero-shot VLMs without fine-tuning are not reliable either: they lack alignment between the modalities.

Adding structured explanations to the OCR text lifts the numbers noticeably: DistilBERT's Macro-F1 on target goes from roughly 56% to 63%, and the correlation on intensity from 26% to 33%. The best results come from TCRNet, which explicitly combines image, text and reasoning:

• dark humor recognition: 75.0% accuracy;
• target prediction: 64.2% weighted F1;
• intensity prediction: 62.7% accuracy, 38.6% Pearson correlation.

The ablation is honest about how much the explanations carry: remove the reasoning stream and Macro-F1 on target falls from about 60.5% to about 35.1%, while weighted F1 on whether the meme is dark humor at all drops from about 74.1% to about 67.3%.

What this changes

The scientific contribution comes down to two things. First, the community gets an open multimodal dataset that clearly labels not only whether a meme is dark humor, but also its target and its intensity. Second, it shows that models get better at subtle memes when they are made to explain the joke and match that explanation against the image and the text. Not just recognizing a pattern, but assembling a causal picture: who is joking, about what, on the strength of which contrast, and why it reads as dark.

Where the limits are

The data comes from Reddit and is in English, so transfer to other platforms and cultures may be uneven, and the distribution across intensity levels is unbalanced. And any model working on sensitive subjects needs responsible usage protocols, from the psychological safety of the annotators to careful control over data access.

Bottom line

D-Humor offers a realistic route to understanding dark humor: add explicit reasoning on top of the multimodal signal and teach the model to align facts with interpretations. That makes the decisions more transparent and more useful, both for research and for applied moderation.

💾 Code