Linear Algebra & Optimization

Matrix Algebra, Rank of a matrix, Reduction to Echelon and Normal forms. Consistency and solution of Systems of Linear Equations (Homogeneous and Non-homogeneous) using Gauss Elimination and Gauss-Jordan methods. LU Decomposition (Algorithmic approach to solving systems). Inverse of a matrix via elementary row operations.

Vector Spaces: Subspaces, Linear Independence/Dependence, Span, Basis, and Dimension. Linear Transformations: Kernel, Range, Rank-Nullity Theorem, and Matrix representation of transformations. Inner Product Spaces: Norms, Orthogonality, Orthogonal Projections, and the Gram-Schmidt Orthogonalization Process (QR Decomposition foundation).

Eigenvalues and Eigenvectors: Characteristic equation, Algebraic and Geometric Multiplicity. Cayley-Hamilton Theorem. Diagonalization of Matrices. Real Symmetric Matrices and Orthogonal Diagonalization. Advanced Factorization: Introduction to Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) (Mathematical foundation for Data Compression and Dimensionality Reduction).

Introduction to Optimization and Convex Sets. Formulation of Linear Programming Problems (LPP). Solution methods: Graphical Method and the Simplex Algorithm (Big-M method, Two-phase method). Duality in Linear Programming: Primal-Dual relationships and Sensitivity Analysis (Economic interpretation of resources).

Unconstrained Optimization: Necessary and sufficient conditions for local extrema. Iterative Search Methods: Gradient Descent (Steepest Descent), Newton-Raphson Method, and Conjugate Gradient Method (Training logic for Neural Networks). Constrained Optimization: Lagrange Multipliers (Revisited) and Karush-Kuhn-Tucker (KKT) conditions for non-linear constraints.