DARE: drop-and-rescale task deltas for model merging

What changed

Randomly removed task-delta entries, rescaled retained entries and combined homologous fine-tuned models with existing merging methods, studying when sparse deltas preserve measured abilities.

Scientific scope

Randomly removed task-delta entries, rescaled retained entries and combined homologous fine-tuned models with existing merging methods, studying when sparse deltas preserve measured abilities. Inherits dropout-style rescaling and model merging. Original source expressly reports failure with larger continued-pretraining deltas; it does not prove SFT never adds knowledge or guarantee arbitrary99percent removal. Published abstract and figure text differ on one merged-model GSM8K value; no exact rate exported.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Inherits dropout-style rescaling and model merging. Original source expressly reports failure with larger continued-pretraining deltas; it does not prove SFT never adds knowledge or guarantee arbitrary99percent removal. Published abstract and figure text differ on one merged-model GSM8K value; no exact rate exported.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction