*X*passing symbols

*x*into a symbol register

_{i}*x*= (

^{n}*x*

_{0},

*x*

_{1}, …,

*x*

_{n-1}). The output symbol can therefore be expressed as a set of compound symbol

*. Therefore*

^{n}X_{C}**Case 1:**If*X*is a DMS then, each*x*are statistically independent. Therefore_{i}*n*= # of individual symbols in*x*, dividing both sides by −1 ∕ 3^{n}*n*→ ∞*n*symbols (*n*trials)*n*→ ∞
❶

**Case 2:**If*X*is**not**a DMS, i.e.,*X*has memory then, its Entropy Rate may be defined as*X*was a DMS**Case 3:**If*X*is an Ergodic source, then

## An Ergodic source

Ergodic source are source whose

- Probabilities don't change with time, i.e., stationary.
- For any and all of its statistics,

Time average = Ensemble average

*What is the practical consequence of AEP? i.e,*

Asymptotic Equipartition Property (AEP) makes **lossless data compression** possible. Lossless data compression is called data **compaction**.

**Reference:**

Cover, T. M., & Thomas, J. A. (2006). Lemma 16.8.1. In 2nd Edition, *Elements of Information Theory* (pp. 646-647). Hoboken, New Jersey: John Wiley & Sons, Inc.

*Next:*

More on AEP (p:2) ➽