Rényi: a family of ways to measure uncertainty and information

What is recorded here?

The1961 chapter distinguishes Shannon entropy from a broader order-parametrized family by changing the mean-value postulate, and develops corresponding information measures under explicit finite-distribution assumptions. An uncertainty measure depends on which rules we ask it to obey. Rényi shows that combining independent experiments additively does not by itself single out Shannon entropy. Requiring an arithmetic mean gives Shannon’s measure; allowing appropriate exponential means produces a family with an adjustable order and Shannon as a limit. The chapter also characterizes corresponding measures of information when one probability distribution is replaced by another. A final finite-state Markov-chain example shows how such information measures can help prove convergence. That example inherits the underlying limit result and the information-proof idea, and does not provide a numerical stopping budget. The contribution is a reusable mathematical framework, rather than one universal uncertainty score that applies unchanged to every setting.

Source-linked catalogue note. A detailed explanatory guide has not yet been prepared for this entry.

Scientific scope

Finite discrete distributions, plus generalized nonnegative weights with 0<W<=1. Independent-product additivity is weaker than Shannon grouping: the order-parametrized entropy family does not inherit every Shannon chain rule. Shannon arises as order tends to 1; exponential versus arithmetic means and stated postulates distinguish the family. Information characterization uses matched finite supports with positive reference probabilities; section 3 explicitly needs an extra continuity postulate to exclude nonpositive orders. The opening Radon–Nikodym discussion is not a proof of every infinite/continuous extension. The Markov illustration assumes finitely many states, strictly positive one-step transitions and an inherited positive stationary distribution; it supplies convergence, not a finite burn-in bound or every irreducible chain theorem. Existing KL and Markov limit results are not claimed as first here. no independent proof, estimator performance, quantum-entropy result or universal measure of scientific value. Earlier 1959/1960 work and 1961 selected edition are distinguished without duplicate entries.

Read the original work

• Rényi, On Measures of Entropy and Information (1961), original547–561
• Rényi Institute author bibliography:1961 entropy chapter entry

This English explanation is an editorial reading view. It does not replace the complete original work or certify its science.

People and systems

Alfréd Rényi

Source-listed contributors to this scoped work. Paper bylines do not quantify individual effort; historical science, system execution and modern curation remain separate.