We announce a course for Ph. D students on “Optimization using Graphical Models”, March 5-14, 2019.

Ph.D. Course in “Optimization using Graphical Models”

S.D.I.A. – Università di Parma

March 5 – 14, 2019

Syllabus

Topic 1: Basics on convex optimization (Locatelli)

Definitions of convex functions and sets. Brief sketches about complexity and algorithms. Constraint qualifications. Optimality conditions (KKT). Lagrangian duality.

Topic 2: Bayesian inference and graphical models (Colavolpe and Vannucci)

Bayesian Networks and Markov Random Fields: Inference in general Graphs.

Variational Inference techniques in Machine Learning and Artificial Intelligence: Belief Propagation (BP) and Loopy Belief Propagation

Factor graphs and sum-product algorithm: general framework and applications to communications.

Topic 3: Variational Inference and the Free Energies methods of Physics (Vannucci)

Approximate Inference and factorized distributions. Variational mixtures of Gaussians. Exponential Family distributions. Variational message passing. Expectation Propagation and its implementation on graphs.

Energy functions and their minimization schemes. Variational average energy and variational entropy; Gibbs and Helmholtz free energies. Stationary conditions for the Bethe free energy and its connections with Loopy Belief Propagation. The Mean Field approximation.

Topic 4: Graphical models for optimization – Trajectory planning and power control (Consolini)

Speed and trajectory planning problems, dynamic programming. Power control in radio systems. Reduction of previous problems to a standard form and definition of the associated graph. Graph-based solution algorithms.

Topic 5: The Physics of Statistical Inference Problems (Caire)

Analysis of inference problems (in particular, posterior-mean estimation and MAP detection) using the Replica method of statistical physics. Decoupled channel model interpretation.

Topic 6: Sparse Signal Reconstruction, Aka Compressed Sensing (Caire)

Problem formulation. Convex relaxation, L1 and LASSO. Performance guarantees. The AMP algorithm and its state evolution analysis. Variations on the theme: multiple measurement vector (MMV) and Vector-AMP (VAMP), Non-Negative Least-Squares, and a new Maximum Likelihood approach. Some applications:

i) channel estimation and pilot decontamination in massive MIMO with correlated antennas;

ii) SPARC codes and their decoding algorithms;

iii) Activity detection and large-scale path loss estimation in massive random access.

Topic 7: Some Relations Between Replica-analysis and Amp State Evolution

(recovering and rigorizing the replica method subject to certain model assumptions).