BWhat changed
Routed late-layer information through a dense CLS vector into an early-token reconstruction head, then added same-document span contrastive pretraining and memory-efficient gradient caching for corpus-aware embeddings.
Scientific scope
Routed late-layer information through a dense CLS vector into an early-token reconstruction head, then added same-document span contrastive pretraining and memory-efficient gradient caching for corpus-aware embeddings. The auxiliary head is removed for fine-tuning; corpus-aware pretraining remains extra work and can require a large effective batch even when task fine-tuning is small-batch. Gradient caching is inherited, not newly invented here. Optimization readiness is a source hypothesis supported by experiments, not a universal formal explanation of retriever training.
Evidence
The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. The auxiliary head is removed for fine-tuning; corpus-aware pretraining remains extra work and can require a large effective batch even when task fine-tuning is small-batch. Gradient caching is inherited, not newly invented here. Optimization readiness is a source hypothesis supported by experiments, not a universal formal explanation of retriever training.
Contributors
- Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.
- Is Your Language Model Ready for Dense Representation Fine-tuning?
- Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.
Full scientific record · Browse contribution grades · Request a correction