Channel capacity. Important particular cases of channels

Abstract

This chapter is devoted to the second variational problem , in which we try to find an extremum of the Shannon’s amount of information with respect to different input distributions. We assume that the channel, i.e. a conditional distribution on its output with a fixed input signal, is known. The maximum amount of information between the input and output signals is called channel capacity. Contrary to the conventional presentation, from the very beginning we introduce an additional constraint concerning the mean value of some function of input variables, i.e. we consider a conditional variational problem. Results for the case without the constraint are obtained as a particular case of the provided general results.