Students are expected to acquire: some knowledge of the main techniques and methods for the solution of numerical and optimization problems; some understanding of the connections between typical techniques of numerical analysis and optimization algorithms; tools for modeling (through numerical analysis and optimization) specific problems from the following areas: regression and parameter estimation in statistics, approximation and data fitting, machine learning, data mining, image and signal reconstruction.
The student will be assessed on her/his demonstrated ability to apply the main course contents to one specific problem, developing an end-to-end application to solve it and a report that describes the application and analyzes critically its perfoirmances. During the oral exam the student must be able to demonstrate her/his knowledge of the course material together with adequate language and proper terminology. Critical awareness of the topics will be also evaluated.
The student will learn to develop and use software to solve some of the basic problems in optimization and linear algebra, including: linear systems and least-squares problems, constrained and non-constrained continuous optimization problems. The student will acquire some knowledge of the theoretical properties of these problems and of the functioning of these algorithms and the ability to correlate the known theoretical properties with the practical behavior of the approaches in order to both check their correctness and possibly improve their efficiency/effectiveness.
The assessment will be largely based on a project work in which the students will be asked to put in practice the learned theory on some specific problem, which (at the choice of the student) may or may not have a relationship with Machine Learning. An oral exam will complement this.
The student will learn to combine analysis and computer implementations to solve the main theoretical problems underlying Data Mining, Artificial Intelligence and Machine Learning. The student will have to learn teamwork strategies, since the projects are supposed to be developed in pairs, as well as proper interaction with the committents of a complex project (the teachers), both vis-a-vis and via e-mail or other electronic means.
The behaviors will be continuously assessed during the development of the project and the final oral exam.
Undergraduate courses in calculus, linear algebra, numerical analysis (recommended) and optimization (recommended).
Delivery: face to face
- attending lectures
- participation in seminars
- individual study
- ICT assisted study
Attendance: Advised, but not strictly necessary. Lectures are recorded and made available to students, which therefore can follow the course even without physically attending the lectures.
Il corso è tenuto in inglese, quindi si prega di fare riferimento al programma in inglese.
- Linear algebra and calculus background
- Unconstrained optimization and systems of equations
- Direct (LU, Cholesky, QR) and iterative (Arnoldi, GMRES, CG) methods for linear systems
- Iterative methods for nonlinear systems
- Numerical methods for unconstrained optimization
- The least-squares problem: solution algorithms and theoretical features (including relations to the singular value decomposition)
- Stability and conditioning of linear equations and least-squares problems
- Constrained optimization
- Duality (Lagrangian, Quadratic, Conic)
- Numerical methods for constrained optimization
- Applications: regression, parameter estimation, approximation and data fitting, support vector machines, signal reconstruction
- Software tools for numerical and optimization problems (Matlab, in particular).
Lecture notes by the lecturers available to students.
- J.W. Demmel "Applied Numerical Linear Algebra" SIAM, 1997
L.N. Trefethen, D. Bau III "Numerical Linear Algebra" SIAM, 1997
J. Nocedal, S.J. Wright "Numerical Optimization" Springer, 1999
- S. Boyd, L. Vandenberghe "Convex optimization" 2004
- M.S. Bazaraa, H.D. Sherali, C.M. Shetty "Nonlinear Programming: Theory and Algorithms" Wiley, 1993
The instructors will both provide slides for all the lessons of the course as well as recordings of the lessons themselves, thereby non-attending students should be able to follow course on par with attending ones.
Project, followed by an oral exam.