This section contains lecture notes and some associated readings.

Complete lecture notes (PDF - 7.7MB)

TOPICS | LECTURE NOTES | READINGS |
---|---|---|

The role of convexity in optimization Duality theory Algorithms and duality | Lecture 1 (PDF - 1.2MB) | |

Convex sets and functions Epigraphs Closed convex functions Recognizing convex functions | Lecture 2 (PDF) | Section 1.1 |

Differentiable convex functions Convex and affine hulls Caratheodory's theorem | Lecture 3 (PDF) | Sections 1.1, 1.2 |

Relative interior and closure Algebra of relative interiors and closures Continuity of convex functions Closures of functions | Lecture 4 (PDF) | Section 1.3 |

Recession cones and lineality space Directions of recession of convex functions Local and global minima Existence of optimal solutions | Lecture 5 (PDF - 1.0MB) | Sections 1.4, 3.1, 3.2 |

Nonemptiness of closed set intersections Existence of optimal solutions Preservation of closure under linear transformation Hyperplanes | Lecture 6 (PDF - 1.4MB) | |

Review of hyperplane separation Nonvertical hyperplanes Convex conjugate functions Conjugacy theorem Examples | Lecture 7 (PDF) | Sections 1.5, 1.6 |

Review of conjugate convex functions Min common / max crossing duality Weak duality Special cases | Lecture 8 (PDF - 1.2MB) | Sections 1.6, 4.1, 4.2 |

Minimax problems and zero-sum games Min common / max crossing duality for minimax and zero-sum games Min common / max crossing duality theorems Strong duality conditions Existence of dual optimal solutions | Lecture 9 (PDF) | Sections 3.4, 4.3, 4.4, 5.1 |

Min common / max crossing Theorem III Nonlinear Farkas' lemma / linear constraints Linear programming duality Convex programming duality Optimality conditions | Lecture 10 (PDF) | Sections 4.5, 5.1, 5.2, 5.3.1–5.3.2 |

Review of convex programming duality / counterexamples Fenchel duality Conic duality | Lecture 11 (PDF) | Sections 5.3.1–5.3.6 |

Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions | Lecture 12 (PDF) | Section 5.4 |

Problem structure Conic programming | Lecture 13 (PDF) | |

Conic programming Semidefinite programming Exact penalty functions Descent methods for convex optimization Steepest descent method | Lecture 14 (PDF) | Chapter 6: Convex Optimization Algorithms (PDF) |

Subgradient methods Calculation of subgradients Convergence | Lecture 15 (PDF) | |

Approximate subgradient methods Approximation methods Cutting plane methods | Lecture 16 (PDF) | |

Review of cutting plane method Simplicial decomposition Duality between cutting plane and simplicial decomposition | Lecture 17 (PDF) | |

Generalized polyhedral approximation methods Combined cutting plane and simplicial decomposition methods | Lecture 18 (PDF) | Bertsekas, Dimitri, and Huizhen Yu. "A Unifying Polyhedral Approximation Framework for Convex Optimization." SIAM Journal on Optimization 21, no. 1 (2011): 333–60. |

Proximal minimization algorithm Extensions | Lecture 19 (PDF) | |

Proximal methods Review of proximal minimization Proximal cutting plane algorithm Bundle methods Augmented Lagrangian methods Dual proximal minimization algorithm | Lecture 20 (PDF - 1.1MB) | |

Generalized forms of the proximal point algorithm Interior point methods Constrained optimization case: barrier method Conic programming cases | Lecture 21 (PDF) | |

Incremental methods Review of large sum problems Review of incremental gradient and subgradient methods Combined incremental subgradient and proximal methods Convergence analysis Cyclic and randomized component selection | Lecture 22 (PDF) | Bertsekas, Dimitri. "Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey." (PDF) Sra, Suvrit, Sebastian Nowozin, and Stephen Wright, eds. |

Review of subgradient methods Application to differentiable problems: gradient projection
Iteration complexity issues Complexity of gradient projection Projection method with extrapolation Optimal algorithms | Lecture 23 (PDF) | |

Gradient proximal minimization method Nonquadratic proximal algorithms Entropy minimization algorithm Exponential augmented Lagrangian method Entropic descent algorithm | Lecture 24 (PDF) | Beck, Amir, and Marc Teboulle. "Gradient-Based Algorithms with Applications to Signal-Recovery Problems." In Beck, Amir, and Marc Teboulle. "Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization." Bertsekas, Dimitri. |

Convex analysis and duality Convex optimization algorithms | Lecture 25 (PDF - 2.0MB) |