Asymptotic Equipartition Property

Fig 1 Symbol Register
Assume an information source X passing symbols xi into a symbol register xn = (x0, x1, …, xn-1). The output symbol can therefore be expressed as a set of compound symbol nXC. Therefore
p(x sup n) = P (x sub 0, x sub 1, ..., x sub n-1)

An Ergodic source

Fig 2 Illustration of ensemble average

Ergodic source are source whose

What is the practical consequence of AEP? i.e, limit as n tends to infinity of (-log p(x sup n)) over n

Asymptotic Equipartition Property (AEP) makes lossless data compression possible. Lossless data compression is called data compaction.


Cover, T. M., & Thomas, J. A. (2006). Lemma 16.8.1. In 2nd Edition, Elements of Information Theory (pp. 646-647). Hoboken, New Jersey: John Wiley & Sons, Inc.


More on AEP (p:2) ➽