TOPICS FOR TEST 3 on Wednesday April 30, 2025

6 Optimization .................................................. 255
    6.1   Optimization Problems ................................. 255
    6.2   Existence and Uniqueness .............................. 258
        6.2.1   Convexity
        6.2.2   Unconstrained Optimality Conditions
        6.2.3   Constrained Optimality Conditions
                Only first order conditions, not second order conditions
    6.3   Sensitivity and Conditioning .......................... 268
    6.4   Optimization in One Dimension ......................... 269
        6.4.1   Golden Section Search
        6.4.2   Successive Parabolic Interpolation                   
        6.4.3   Newton's Method
        6.4.4   Safeguarded Methods
    6.5   Unconstrained Optimization ............................ 275
        6.5.1   Direct Search                                        
        6.5.2   Steepest Descent
        6.5.3   Newton's Method
        6.5.4   Quasi-Newton Methods
        6.5.5   Secant Updating Methods (time & space costs)
        6.5.6   Conjugate Gradient Method (time & space costs)
        6.5.7   Truncated or Inexact Newton Methods
    6.6   Nonlinear Least Squares ............................... 284
        6.6.1   Gauss-Newton Method                                  
        6.6.2   Levenberg-Marquardt Method                           
    6.7   Constrained Optimization .............................. 287
        6.7.1   Sequential Quadratic Programming                     
        6.7.2   Penalty and Barrier Methods
        6.7.3   Linear Programming