Overview of our EC Pretraining Framework. A speaker-listener pair engages in a referential game, where the speaker generates an Emergent Communication (EC) message to describe a target image, and the listener must identify the correct image among distractors. The resulting EC tokens serve as pretraining supervision for a Vision-Language Model (VLM). This EC-pretrained VLM is then fine-tuned on a range of downstream vision-language tasks, including Visual Entailment, Visual Referring Expression, Image Captioning, and Visual Question Answering. The framework enables transferable visual grounding from synthetic EC messages to natural language tasks.
-
Referential game
The speaker describes a target image through EC tokens. The listener uses the message to choose that image among distractors.
-
EC pretraining
Image–message pairs provide visually grounded supervision for a vision-language model, without human-written pretraining captions.
-
Downstream transfer
The pretrained model is fine-tuned for natural-language tasks, including entailment, referring expressions, captioning, and visual question answering.

