Hours:
16 hours (4 credits)

Room:

Aula Riunioni del Dipartimento di Ingegneria dell’Informazione, Via G. Caruso 16, Pisa - Ground Floor

To register to the course, click here

Short Abstract:

Optimization is central to solving problems in many engineering disciplines. Specifically in information engineering, the areas of signal and image processing, communications and radar systems, networking, control and robotics, and power systems benefit immensely from optimization theory and algorithms. Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions (or, equivalently, maximizing concave functions) over convex sets. This course will introduce essential elements of theory of convex optimization. While some problems may be solved analytically, often the rich body of numerical algorithms is needed in most practical settings. A rigorous coverage will be provided in best-known foundational numerical techniques that can be leveraged by students to understand and engage with the ever evolving body of numerical methods in the literature. Finally, applications will be discussed in signal processing and machine learning as compelling real-world problems.

Course Contents in brief:

Day 1 ( 4 hours)

Part 0: Linear Algebra Review

Part I: Theory

1. Introduction to mathematical optimization, cost function development, identification of key issues in formulating and solving optimization problems

2. Convex Sets: Key examples, convexity preserving operations, inequalities of interest – Jensen’s

3. Convex Functions: Properties and examples, convexity preserving operations, important convex functions, quasi convex functions and approximating non-convex functions

Day 2 (4 hours)

4. Convex Optimization Problems: Unconstrained optimization, linear and quadratic forms, geometric programming, semi-definite programming, formulation of important known problems as semi-definite programs and corresponding key results

5. Constrained convex optimization: Identifying a convex problem, linear and non-linear constraints, equality and inequality constraints. Lagrange dual function, dual problem, optimality conditions (KKT).

Day 3 (4 hours)

Part II: Algorithms

1. Unconstrained minimization: descent methods, gradient descent, steepest descent, Newton’s method, and quasi-Newton and BFGS methods, implementation concerns and tricks.

2. Constrained minimization: Newton’s method with equality constraints, Infeasible start, implementation concerns

3. Interior Point Methods: Inequality constrained problems, log barrier function and central path, the barrier method, problems with generalized inequalities, primal-dual interior point methods.

4. Optimization software: MATLAB optimization toolbox and other key packages.

Day 4 (4 hours)

Part III: Real-world optimization problems with applications

1. Sparsity Constrained Optimization and Estimation

2. Covariance Estimation with structural constraints

Applications in statistics, machine learning and radar signal processing.

Schedule:

1. Day1 – 9 AM – 1 PM
2. Day2 – 9 AM-1PM
3. Day3 – 9 AM-1 PM
4. Day4 – 9 AM – 1 PM
5. Day5 – 10 AM-1 PM (final written exam)