Course Material for AI2100: Convex Optimization (Spring 2024)

Lecture Notes and Classroom Scribble

The classroom scribble and the lecture notes are intended for quick reference\review. Lecture notes and classroom scribble are by no means a complete source and infact are based on the suggested textbooks\references. It is highly recommended that the students also refer to the suggested textbooks\references.

Lec. (Date)	Topics Covered	Logistics
01 (01 Jan)	Introduction Course overview
02 (02 Jan)	Single Variable Optimization [ Notes ] Overview of concepts in Single Variable Optimization
03 (05 Jan)	Algorithms for Single Variable Optimization [ Scribble \| Notes ] Overview of Search Space Reduction Techniques Introduction to Golden Section Search
04 (08 Jan)	Algorithms for Single Variable Optimization [ Scribble \| Notes ] Golden Section Search
05 (09 Jan)	Algorithms for Single Variable Optimization [ Scribble \| Notes ] The Bisection Search
06 (12 Jan)	Algorithms for Single Variable Optimization [ Scribble \| Notes ] Newton's Method for Unconstrained Single Variable Optimization
07 (16 Jan)	Algorithms for Single Variable Optimization [ Scribble \| Notes ] Secant Method for Unconstrained Single Variable Optimization Newton's method for solving Nonlinear Equations
08 (18 Jan)	Linear and Affine Combination [ Scribble \| Notes ] Linear and Affine Combination Subspace and Affine Hull	HW01 out (20 Jan)
09 (22 Jan)	Affine and Convex Combination [ Scribble \| Notes ] Affine Sets and their Properties Convex Hull
10 (29 Jan)	Convex Sets and Convex Hull [ Scribble \| Notes ] Convex Sets and their properties Algorithms for finding the convex hull	HW01 due (30 Jan)
11 (01 Feb)	Function of Multiple Variables [ Scribble \| Notes ] Function of Multiple Variables Linear and Affine Functions	HW02 out (07 Feb) HW03 out (09 Feb)
12 (12 Feb)	Convex Functions [ Scribble \| Notes ] Convex Functions Review of Limit and Derivative of a function of single variable
13 (15 Feb)	Derivative of a function of single variable [ Scribble \| Notes ] Derivative as an existence of affine approximation
14 (16 Feb)	Derivative of a function of Multiple Variables [ Scribble \| Notes ] Derivative of a function of Multiple Variables Gradient, Hessian, and Jacobian	HW04 out (18 Feb)
15 (19 Feb)	Directional Derivative and the Gradient [ Scribble \| Notes ] Approximations and the Chain Rule Directional Derivative and the Gradient	HW02 due (19 Feb) HW03 due (19 Feb)
16 (20 Feb)	Conditions for Convexity [ Scribble \| Notes ] Feasible Directions First and Second Order Conditions for Convexity
17 (23 Feb)	Optimality Conditions [ Scribble \| Notes ] Taylor's Series Expansion First and Second Order Conditions for Optimality	HW05 out (23 Feb)
18 (26 Feb)	Optimality Conditions [ Scribble \| Notes ] Second Order Sufficient Condition for Optimality General Framework of Optimization Algorithms
19 (27 Feb)	Linear Programming and Newton's Method [ Scribble \| Notes ] Overview of Linear Programming Newton's Method	HW04 due (28 Feb)
20 (01 Mar)	Solving System of Nonlinear Equations [ Scribble \| Notes ] Quadratic Forms Newton's Method for Solving System of Nonlinear equations	HW06 out (03 Mar)
21 (04 Mar)	State Estimation [ Scribble \| Notes ] Overview of State Estimation Gauss Newton Method	HW05 due (04 Mar)
22 (05 Mar)	Descent based Approaches [ Scribble \| Notes ] Descent Direction Descent Direction based Approaches to Compute Optima
23 (08 Mar)	Gradient Descent Approach [ Scribble \| Notes ] Gradient Descent Approach An interesting Example	HW07 out (09 Mar)
24 (11 Mar)	Steepest Descent Algorithm [ Scribble \| Notes ] Steepest Descent Approach Use of Steepest Descent to Solve System of Linear Equations
25 (13 Mar)	Other Descent based Approaches [ Scribble \| Notes ] Newton's Method as a Version of the Gradient Descent	HW06 due (13 Mar)
26 (15 Mar)	Conjugate Directions and Linear Equations [ Scribble ] Conjugate Directions Using Conjugate Directions to solve a system of Sparse Symmetric Linear Equations	HW08 out (16 Mar)
27 (18 Mar)	Conjugate Gradient and Linear Equations [ Scribble ] Conjugate Gradient Using Conjugate Gradient to solve a system of Sparse Symmetric Linear Equations
28 (19 Mar)	Optimization with Constraints [ Scribble \| Notes ] Overview of Optimization Problems with Constraints Intutional understanding of Lagrange Condition	HW07 due (19 Mar)
29 (22 Mar)	Optimization with Constraints [ Scribble \| Notes ] Intutional understanding of Lagrange Condition First and Second order Optimality Conditions	HW09 out (26 Feb)
30 (01 Apr)	Lagrange Condition [ Scribble \| Notes ] Regular Point Lagrange Condition
31 (02 Apr)	Lagrange Condition [ Scribble \| Notes ] Proof of Lagrange Condition
32 (05 Apr)	Optimization with Inequality Constraints [ Scribble \| Notes ] Introduction to optimization Problems with Inequality Constraints	HW10 out (07 Feb) HW08 due (08 Apr)
33 (09 Apr)	Optimization with Inequality Constraints [ Scribble \| Notes ] KKT Conditions Examples
34 (12 Apr)	Optimization with Inequality Constraints [ Scribble \| Notes ] Proof of Sufficiency of KKT Conditions
35 (16 Apr)	Optimization with Constraints [ Scribble \| Notes ] Examples
		HW09 due (04 May) HW10 due (04 May)