Computation of Generalized (h, Φ)-Entropies for Denumerable

Shannon [1948] adapted to the field of information theory the concept of entropy introduced by Boltzmann and Gibbs in 19th century. Since then many different generalized entropies have been defined to adapt to many different fields. Keeping this in mind, in this paper, some new (h, f)-entropies of random sequences-especially Markov chains, taking values in countable spaces, either finite or denumerable, are suggested. The entropy of the stationary distribution of a Markov chain is the (asymptotic) entropy of the chain at equilibrium; if this distribution is taken as initial distribution of the chain, its entropy is also the marginal entropy of the chain. In both cases, the entropy rate is more representative of the whole trajectory of the sequence. Having the marginal entropy and entropy rate of Markov chains under an explicit form allows one to use them efficiently in all applications involving Markov modeling. When only observations of the chain are available, the need for estimation obviously appears. The case of countable parametric chains, for which transition probabilities are functions of a finite set of parameters, is considered. Further, since the entropy is an explicit function of the transition probabilities and hence, of the parameters, plug-in estimators of the marginal entropy and entropy rate can be obtained by replacing the parameters by their maximum likelihood estimators (MLE). This suggests that there is scope for further research, which can be accomplished by computing and estimating the generalized entropy rates for these computed (h, f)-entropies. Along with this in the last part of this paper, ten new functions are computed using classical entropies, which can further be adapted in the field of entropies for denumerable Markov chains.