The variational Bayes approach in signal processing

Abstract

This thesis is concerned with Bayesian identification of parameters of linear models. Linear models are used in many important problems of Digital Signal Processing (DSP). Computationally efficient methods of parameter inference are available under certain restrictive assumptions, such as known transformation of system output, known number of signal sources, etc. These assumptions, however, limit the applicability of these models. In this thesis, we study four important special cases of the linear model as listed below. When we relax the restrictive assumption, in each case, the Bayesian inference becomes intractable. Tractability is restored using the Variational Bayes (VB) approximation technique. Special attention is paid to computational efficiency and flow of control of the associated inference algorithms. Chapters 2 and 3 review the relevant state-of-the-art knowledge. In Chapter 2, the basics of Bayesian parameter inference, and the most common approximation techniques, are reviewed. The Variational Bayes (VB) method is chosen as a reasonable trade-off between accuracy and computational requirements. In Chapter 3, the linear model is introduced and existing Bayesian inference methods are reviewed for this context. At the end of Chapter 3, in Section 3.5, four special cases of the linear model are selected for detailed consideration in the rest of the thesis. For each of these models, a computationally efficient Bayesian inference technique is not currently available and the aim of the thesis is to derive one.