RELATE will be undergoing maintenance starting 9pm on Saturday March 30. It will be available again Sunday morning 6am at the latest. Please plan your work accordingly.

CS450 Calendar

Note: Some calendar entries are clickable and link to entries below.
Lecture 29 (Dec. 7, 2021, 11 a.m. - Dec. 7, 2021, 12:15 p.m.)

Quiz for Lecture 29

  • BVP conditioning, solution uniqueness
  • Shooting method, finite difference method
  • Collocation method
  • Weak solutions, weighted residual method
  • Galerkin/finite element method

Scribbles

Lecture 28 (Dec. 2, 2021, 11 a.m. - Dec. 2, 2021, 12:15 p.m.)

Quiz for Lecture 28

  • Runge-Kutta methods: Heun's method, RK4
  • Multi-step methods (BDFs)
  • Stability regions
  • More advanced methods for IVPs
  • BVP problem setup, solution existence

Scribbles

Lecture 27 (Nov. 30, 2021, 11 a.m. - Nov. 30, 2021, 12:15 p.m.)

Quiz for Lecture 27

  • Stability of a method
    • Forward Euler
    • Systems
    • Nonlinear ODEs
    • Backward Euler
  • Stiff ODEs
  • Predictor-corrector methods
  • Runge-Kutta methods, Butcher tableau, properties

Scribbles

Lecture 26 (Nov. 18, 2021, 11 a.m. - Nov. 18, 2021, 12:15 p.m.)

Quiz for Lecture 26

  • Properties of ODEs: linear, homogeneous, constant-coefficient, autonomous
  • Existence and uniqueness, conditioning
  • Euler's method (forward and backward)
  • Global and local error, order of accuracy
  • Stability of a method

Scribbles

Lecture 25 (Nov. 16, 2021, 11 a.m. - Nov. 16, 2021, 12:15 p.m.)

Quiz for Lecture 25

  • Differentiation matrices
  • Finite differences and Taylor expansion
  • Richardson extrapolation and Romberg integration
  • Examples of IVPs and BVPs
  • IVP for ODEs, reducing ODEs to first-order form

Scribbles

Lecture 24 (Nov. 11, 2021, 11 a.m. - Nov. 11, 2021, 12:15 p.m.)

Quiz for Lecture 24

  • Gaussian quadrature continued
  • Composite quadrature
  • Numerical differentiation
  • Finite differences (forward difference)

Scribbles

Lecture 23 (Nov. 9, 2021, 11 a.m. - Nov. 9, 2021, 12:15 p.m.)

Quiz for Lecture 23

  • Quadrature, conditioning
  • Interpolatory quadrature
    • Newton-Cotes
      • Midpoint rule
      • Trapezoidal rule
      • Simpson's rule
      • Simpson's second (3/8) rule
    • Clenshaw-Curtis
  • Accuracy and stability of interpolatory quadrature
  • Gaussian quadrature

Scribbles

Lecture 22 (Nov. 4, 2021, 11 a.m. - Nov. 4, 2021, 12:15 p.m.)

Quiz for Lecture 22

  • Truncation error in interpolation
  • Connection to Chebyshev nodes, convergence of order $n$
  • Piecewise linear interpolation
  • Piecewise cubic interpolation (splines)

Scribbles

Lecture 21 (Nov. 2, 2021, 11 a.m. - Nov. 2, 2021, 12:15 p.m.)

Quiz for Lecture 21

  • Modes and nodes
  • Lagrange interpolation
  • Newton interpolation
  • Orthogonal polynomials
    • Legendre polynomials
    • Chebyshev polynomials
  • Chebyshev interpolation

Scribbles

Lecture 20 (Oct. 28, 2021, 11 a.m. - Oct. 28, 2021, 12:15 p.m.)

Quiz for Lecture 20

  • Gauss-Newton and Levenberg-Marquardt
  • Constrained optimization
    • Lagrange multipliers
    • KKT conditions
  • Intro to Interpolation

Scribbles

Lecture 19 (Oct. 26, 2021, 11 a.m. - Oct. 26, 2021, 12:15 p.m.)

Quiz for Lecture 19

  • Improving steepest descent: momentum, heavy ball method
  • Brief introduction to: SGD, ADAM, conjugate gradient method, Nelder-Mead method
  • Newton's method and quasi-Newton methods ($n$ dimension)
  • Nonlinear least squares: setup and Gauss-Newton method

Scribbles

Lecture 18 (Oct. 21, 2021, 11 a.m. - Oct. 21, 2021, 12:15 p.m.)

Quiz for Lecture 18

  • Convexity
  • Optimality conditions
  • Sensitivity and conditioning
  • Unimodality, golden section search
  • Newton's method
  • Steepest descent, convergence

Scribbles

Lecture 17 (Oct. 19, 2021, 11 a.m. - Oct. 19, 2021, 12:15 p.m.)

Quiz for Lecture 17

Topics:

  • Discussion on Newton's method
  • Muller's method, inverse quadratic interpolation
  • Strategies to achieve global convergence
  • Systems in $n$ dimension (fixed point iteration, Newton's method, Broyden's method)
  • Optimization: problem statement
  • Optimization: existence and uniqueness, coercivity

Scribbles

Lecture 16 (Oct. 14, 2021, 11 a.m. - Oct. 14, 2021, 12:15 p.m.)

Quiz for Lecture 16

Topics:

  • Stopping criteria
  • Bisection method
  • Fixed point iteration
  • Newton's method
  • Secant method

Scribbles

Lecture 15 (Oct. 12, 2021, 11 a.m. - Oct. 12, 2021, 12:15 p.m.)

Quiz for Lecture 15

Topics:

  • SVD (kiddy version)
  • Nonlinear equations, existence of a root
  • Sensitivity and multiplicity
  • Rates of convergence

Scribbles

Lecture 14 (Oct. 7, 2021, 11 a.m. - Oct. 7, 2021, 12:15 p.m.)

Quiz for Lecture 14

Topics:

  • Computational cost of QR iteration
  • QR iteration with Hessenberg matrices
  • Krylov space methods, Arnoldi iteration and Lanczos algorithms
  • Ritz values

Scribbles

Lecture 13 (Oct. 5, 2021, 11 a.m. - Oct. 5, 2021, 12:15 p.m.)

Quiz for Lecture 13

Topics:

  • Schur form in details
  • Eigenvalues
  • Eigenvectors
  • Computing multiple eigenvalues
  • Simultaneous iteration
  • Orthogonal iteration
  • QR iteration (with shift)

Scribbles

Lecture 12 (Sept. 30, 2021, 11 a.m. - Sept. 30, 2021, 12:15 p.m.)

Quiz for Lecture 12

Topics:

  • In practice
    • Power iteration
    • Inverse iteration
    • Rayleigh quotient iteration
    • Convergence of power iteration
    • Failure modes of power iteration
    • Cost of power iteration
  • Schur form

Scribbles

Lecture 11 (Sept. 28, 2021, 11 a.m. - Sept. 28, 2021, 12:15 p.m.)

Quiz for Lecture 11

Topics:

  • Similar matrices, diagonalizability
  • Sensitivity of eigenvalues
  • Power iteration, inverse iteration, Rayleigh quotient iteration

Scribbles

Lecture 10 (Sept. 23, 2021, 11 a.m. - Sept. 23, 2021, 12:15 p.m.)

Quiz for Lecture 10

Topics:

  • Rank-deficient matrix and least-squares
  • SVD, Eckart-Young-Mirsky theorem
  • Total least-squares, pseudoinverse (rank-deficient generalization)
  • Eigenvalues, algebraic and geometric multiplicity, defective matrix

Scribbles

Lecture 9 (Sept. 21, 2021, 11 a.m. - Sept. 21, 2021, 12:15 p.m.)

Quiz for Lecture 9

Topics:

  • Solve linear least squares using QR
  • (Modified) Gram-Schmidt
  • Householder reflectors
  • Givens rotations

Scribbles

Lecture 8 (Sept. 16, 2021, 11 a.m. - Sept. 16, 2021, 12:15 p.m.)

Quiz for Lecture 8

Topics:

  • Data fitting with polynomials
  • Least squares
  • Normal equation, orthogonal projection, pseudoinverse
  • Sensitivity and conditioning of least squares
  • QR factorization (the very beginning)

Scribbles

Lecture 7 (Sept. 14, 2021, 11 a.m. - Sept. 14, 2021, 12:15 p.m.)

Quiz for Lecture 7

Topics:

  • Partial and complete pivoting
  • BLAS and LAPACK, LU complexity
  • LU for non-invertible or rectangular matrices
  • Round-off error in LU and pivoting
  • Schur complement, Sherman-Morrison formula

Scribbles

Lecture 6 (Sept. 9, 2021, 11 a.m. - Sept. 9, 2021, 12:15 p.m.)

Quiz for Lecture 6

Topics:

  • Error when changing the matrix, Left/right preconditioning
  • SVD, relation to 2-norm and Frobenius norm
  • Back-substitution, Gaussian elimination, LU factorization

Scribbles

Lecture 5 (Sept. 7, 2021, 11 a.m. - Sept. 7, 2021, 12:15 p.m.)

Quiz for Lecture 5

Topics:

  • Linear system, matrix norms
  • Condition number of a matrix
  • Residual and error

Scribbles

Examlet 0 (tentative) (Sept. 2, 2021 - Sept. 5, 2021)

As you may have seen in our class policies, our "examlets" will take place in a computer-based testing facility ("CBTF") in Grainger Library.

You must schedule your test appointment with the Computer-Based Testing Facility at this link.

Find out more about the testing facility, such as:

  • where it is
  • when to show up
  • what to bring (and not to bring)

The exam will be 110 minutes in length, and it will cover the material of the first two lectures, plus linear algebra basics and Python coding basics.

Lecture 4 (Sept. 2, 2021, 11 a.m. - Sept. 2, 2021, 12:15 p.m.)

Quiz for Lecture 4

Topics:

  • Rounding modes, density of floating point numbers
  • Unit roundoff (machine epsilon)
  • Floating point arithmetics

Scribbles

Lecture 3 (Aug. 31, 2021, 11 a.m. - Aug. 31, 2021, 12:15 p.m.)

Quiz for Lecture 3

Topics:

  • Floating point
  • Fw/bw error in-class activity
  • Underflow, overflow, rounding, subnormals

Scribbles

Lecture 2 (Aug. 26, 2021, 11 a.m. - Aug. 26, 2021, 12:15 p.m.)

Quiz for Lecture 2

Topics:

  • Forward/backward error
  • Conditioning
    • Example: Conditioning of function evaluation
  • Accuracy and stability

Scribbles

Lecture 1 (Aug. 24, 2021, 11 a.m. - Aug. 24, 2021, 12:15 p.m.)

Topics:

  • Introduction
  • Class Policies
  • Truncation and Rounding
  • Well-posedness
  • Norms (the very beginning)

Scribbles