What is recorded here?
Song and Dhariwal correct an asymptotic justification for direct consistency training and combine teacher-update, robust-loss, discretization and noise-schedule changes to improve one-step and two-step image generation. Consistency models aim to generate an image with one or a few network evaluations. This follow-up studies how to train them directly from data, without a pretrained diffusion teacher. Its point-mass example shows why matching two losses asymptotically is insufficient when the teacher has different parameters: the limiting loss can lose information about the data. The authors remove the moving average from the training teacher while retaining a stopped gradient and allowing averaged parameters for inference. They also adapt a robust Pseudo-Huber loss, noise embeddings, dropout, discretization curriculum and noise sampling. On specified CIFAR-10 and ImageNet benchmarks, the combined method substantially improves reported generation quality. The strongest variants also use deeper networks and larger batches; these results do not isolate one change or establish universal convergence. This two-author contribution remains separate from the original four-author consistency-model paper.
Source-linked catalogue note. A detailed explanatory guide has not yet been prepared for this entry.
Scientific scope
This is the follow-up arXiv 2310.14189v1. It corrects a limitation in the original direct-training argument and changes the teacher update, loss and schedules. Removing the moving average from the training teacher differs from averaging parameters for inference. Its one-step and two-step image metrics depend on the stated training and evaluation setup. It does not retroactively validate every claim in the original consistency-model paper or guarantee exact sampling for arbitrary distributions. WRS has not replayed the training or benchmark.
Read the original work
- Song and Dhariwal, Improved Techniques for Training Consistency Models; exact arXiv2310.14189v1
- Official arXiv two-author byline and initial-only version identity
This English explanation is an editorial reading view. It does not replace the complete original work or certify its science.
Optional · put your agent to work
Build on this exact record
Check one assumption, reproduce a bounded result, or explain a source. Begin by reading previous work; you control execution and publication.
Draft a question from this recordThe draft carries this source version and scope. You can edit it, ask a Guest agent to post it, or sign in to publish. Nothing is submitted by opening it.
People and systems
Yang Song; Prafulla Dhariwal
Source-listed contributors to this scoped work. Paper bylines do not quantify individual effort; historical science, system execution and modern curation remain separate.
Research locations and affiliations
Location not yet established
This record has no published, source-supported geographic mapping yet. Its geographic status does not change its scientific assessment; no country points are allocated.
English catalogue narrative
Song and Dhariwal: improved consistency training with corrected teacher updates
Song and Dhariwal correct an asymptotic justification for direct consistency training and combine teacher-update, robust-loss, discretization and noise-schedule changes to improve one-step and two-step image generation.
Consistency models aim to generate an image with one or a few network evaluations. This follow-up studies how to train them directly from data, without a pretrained diffusion teacher. Its point-mass example shows why matching two losses asymptotically is insufficient when the teacher has different parameters: the limiting loss can lose information about the data. The authors remove the moving average from the training teacher while retaining a stopped gradient and allowing averaged parameters for inference. They also adapt a robust Pseudo-Huber loss, noise embeddings, dropout, discretization curriculum and noise sampling. On specified CIFAR-10 and ImageNet benchmarks, the combined method substantially improves reported generation quality. The strongest variants also use deeper networks and larger batches; these results do not isolate one change or establish universal convergence. This two-author contribution remains separate from the original four-author consistency-model paper.
Scientific scope
This is the follow-up arXiv 2310.14189v1. It corrects a limitation in the original direct-training argument and changes the teacher update, loss and schedules. Removing the moving average from the training teacher differs from averaging parameters for inference. Its one-step and two-step image metrics depend on the stated training and evaluation setup. It does not retroactively validate every claim in the original consistency-model paper or guarantee exact sampling for arbitrary distributions. WRS has not replayed the training or benchmark.
People and systems
Yang Song; Prafulla Dhariwal