Gallagher's diagram and Entropy rate

## Gallagher's diagram

An alternative to the Markov process diagram is the Gallagher's diagram. The diagram represents the source and channel using state-dependent memoryless channel diagrams.

For our above example, below-left shows the Gallagher's diagram is at time index n−1 the state is S0 while the right hand side shows the diagram if at n−1 the state is S1.

Note that the output y(n) and state S(n) [or S(n) for short] updates simultaneously. That is

but

From Gallagher's diagram we have

In other words,

Since

but we can therefore say that y(n) and S(n) are state independent.

## Entropy rate

Let index i = 0 or 1 but fixed. Then for a fixed S(n−1) we can get the conditional entropy H(Y | S(n−1) = Si). That is

Recall that the elements of the F matrix are fij = p(yj | Si). Thus Also recall that for a fixed Si, the elements fij goes down the respective column in F. That is In our example system

Hence, for fixed state Si = S0

and for fixed state Si = S1 Note that the above computed values of unity are labels for arrows seen in the Gallagher's diagram.

Following determination of the conditional probability H(Y | S(n−1) = Si), the steady-state entropy rate of the output sequence is therefore

In general, the entropy rate is given by