E1 251-O Linear and Non-linear Optimization 3:0 (January 2022)

Course Instructor: Chandramani Singh, ESE

Course description: In this course, we will study the basics of linear and nonlinear optimization. We will also see several usages of optimization techniques in supervised and unsupervised learning.

Syllabus

SOptimization examples, The basics - global vs local optimality; Convex sets, Convex and concave functions; First-order and second-order optimality conditions; Gradient descent methods, Conjugate gradient method, Newton method, Gradient projection method; Constrained optimization with equality and inequality constraints, Duality; Linear programming, simplex method, duality; Barrier and penalty function methods; Subgradient descent methods; Proximal gradient descent; Augmented Lagrangian methods.

Textbooks / References

  1. D. Bertsekas. Nonlinear Programming, Athena Scientific, 2016.
  2. D. Luenberger and Y. Ye. Linear and Nonlinear Programming, Springer, 2008.
  3. S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004

Prerequisites: None

Grading:

  • Homeworks(assigned approximately once in two weeks) 40%
  • Midterm 30%
  • Final exam 30%.

Homeworks will be assigned approximately once in two weeks, and a scanned copy of the solutions need to be turned in.