Mutual Information Rate Decomposition as a Tool to Investigate Coupled Dynamical Systems: Estimation Approaches, Simulations and Application to Physiological Signals

Abstract

In this work, we present a framework for the computation of the MIR between two random processes X and Y, expressed equivalently as the sum of the individual entropy rates of X and Y minus their joint entropy rate, or as the sum of the transfer entropies from X to Y and from Y to X plus the instantaneous information shared by the processes at zero lag. After defining the theoretical formulation of the framework, different approaches for the estimation of each dynamic measure composing the MIR are provided: the linear model-based estimator relying on Gaussian data; two model-free estimators based on discretization, performed via uniform quantization through binning or rank ordering through permutations; a model-free estimator based on direct computation of the differential entropy via k-nearest neighbor searches.