Caron et al.: label-free momentum-teacher distillation for visual transformers

What changed

Adapted label-free teacher–student self-distillation, momentum targets and multi-crop training to visual transformers, producing transferable features and emergent object-region attention.

Scientific scope

Adapted label-free teacher–student self-distillation, momentum targets and multi-crop training to visual transformers, producing transferable features and emergent object-region attention. Momentum teachers, crops, transformers and distillation have predecessors. Object-like attention maps are not a supervised segmentation guarantee. Original DINO is distinct from later DINOv2/3 resource and training improvements.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Momentum teachers, crops, transformers and distillation have predecessors. Object-like attention maps are not a supervised segmentation guarantee. Original DINO is distinct from later DINOv2/3 resource and training improvements.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• Emerging Properties in Self-Supervised Vision Transformers

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction