RepoBench: cross-file retrieval and repository-level code completion

What changed

Constructed repository-separated Python and Java tasks for cross-file snippet retrieval, next-line completion and an integrated retrieval-completion pipeline, distinguishing first-use and repeated-use contexts.

Scientific scope

Constructed repository-separated Python and Java tasks for cross-file snippet retrieval, next-line completion and an integrated retrieval-completion pipeline, distinguishing first-use and repeated-use contexts. Repository-level splitting reduces direct within-dataset overlap but does not establish absence from model pretraining. Retrieval and completion metrics are separate; next-line completion is not autonomous repository repair.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Repository-level splitting reduces direct within-dataset overlap but does not establish absence from model pretraining. Retrieval and completion metrics are separate; next-line completion is not autonomous repository repair.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction