Μαθήματα

Welcome to grad course 602! This is a core graduate course, useful for people specialising in various aspects of engineering.
Throughout the semester, we derive and discuss basic theoretical tools and provide concrete examples. Even though practical
engineering problems will be analyzed, the course aims to develop solid problem solving skills applicable to more general settings.

It is hard to find a problem in digital communications, big data analytics, pattern recognition, radar/satellite engineering, bioinformatics, i.e., the broad area of signal processing that does not involve core detection and estimation theory!
Students must feel comfortable with probability and linear algebra; however, all necessary background material is taught in the class.

Pls consult "Extended Syllabus" in the "Documents" section for course content.

Supervised learning: least mean squares (LMS), logistic regression, perceptron, Gaussian discriminant analysis, naive Bayes, support vector machines, model selection and feature selection, ensemble methods (bagging, boosting). Deep Neural Networks & Modern Tools (e.g., Tensorflow, AutoML). Generative Adversarial Neural Networks (GANN) & Applications. Learning theory: bias/variance tradeoff, union and Chernoff/Hoeffding bounds, VC dimension. Unsupervised learning: clustering, k-means, EM, mixture of Gaussians, factor analysis, principal components analysis (PCA), independent component analysis (ICA). Reinforcement learning: Markov decision processes (MDPs), algorithms for POMDPs.

Course topics

Supervised learning: least mean squares (LMS), logistic regression, perceptron, Gaussian discriminant analysis, naive Bayes, support vector machines, model selection and feature selection, ensemble methods (bagging, boosting). Deep Neural Networks & Modern Tools (e.g., Tensorflow, AutoML). Generative Adversarial Neural Networks (GANN) & Applications. Learning theory: bias/variance tradeoff, union and Chernoff/Hoeffding bounds, VC dimension. Unsupervised learning: clustering, k-means, EM, mixture of Gaussians, factor analysis, principal components analysis (PCA), independent component analysis (ICA). Reinforcement learning: Markov decision processes (MDPs), algorithms for POMDPs.

Course Syllabus

Course Outline

Course Introduction:

This course introduces the student to the theory, techniques and algorithms used in machine learning. Specifically, the course offers a comprehensive overview of modern methods related to machine learning and pattern recognition, with a focus on how these techniques are applied in various applications such as digital images, human speech, text processing, etc. Understanding the theory is achieved through a combination of theoretical and practical laboratory exercises.

Introduction to Pattern Recognition and Machine Learning

General principles and applications – Classification and Regression – Supervised, Unsupervised, Reinforcement Learning – The design cycle of a pattern recognition system – Feature selection and modeling – Learning and adaptation.

Feature Selection and Transformations

Feature selection and normalization – Feature transformations:

• Principal Component Analysis (PCA)

• Independent Component Analysis (ICA)

• Linear Discriminant Analysis (LDA)

• DFT & other frequency domain transformations.

Bayesian Decision Theory

Bayes classes and classifiers – Bayesian decision theory for continuous vectors – Loss function and risk – Discriminant functions, decision surfaces – Minimum error classification – Examples with two classes and normal distributions – Classification error probability – Bayesian decision theory for discrete vectors.

Parameter Estimation and Supervised Learning

Maximum Likelihood (ML) estimation – General Bayesian estimation – Maximum a posteriori (MAP) estimation – Bayes learning of the mean of a normal distribution – Sufficient Statistics – Exponential family of distributions.

Linear Classifiers

Designing linear classifiers – Linear discriminant functions and decision surfaces – Linearly separable classes – Gradient descent rule – Perceptron algorithm.

Linear Regression

Methods for minimizing the mean squared error (MSE) – Logistic regression.

Nonlinear Classifiers, Perceptron, and Artificial Neural Networks (ANNs)

Introduction to neural networks – The simple Perceptron – Multi-layer Perceptron – Feedforward Neural Networks – Backpropagation algorithm.

Deep Learning, Deep Neural Networks (DNNs)

Deep Neural Networks (DNN) – Recurrent Neural Networks (RNN) – Learning Deep Neural Networks – Examples.

Unsupervised Learning and Clustering

Data description and clustering – Distance metrics – k-means algorithm – Hierarchical clustering – Nearest neighbor decision algorithm – Probabilistic clustering (Gaussian Mixture Models) – Expectation-Maximization (EM) algorithm.

Non-parametric classification methods

Distribution estimation – Parzen windows – Support Vector Machines (SVM) – Boosting algorithm – k-Nearest Neighbors (k-NN) method – Classification and regression trees.

Applications

Phase portrait. Second-order systems. Existence and uniqueness of solutions. Sensitivity equations. Comparison principle. Lyapunov stability. LaSalle theorem. Linearization. Center manifold theorem. Stability of perturbed systems. Input-to-state stability. Input-output stability. Perturbation theory and averaging. Singular perturbations. Circle and Popov criteria. Nonlinear control design using backstepping.

State vector Hilbert spaces, Qubit. Theory of quantum measurements, orthonormal complete bases, projectors, positive operator-valued probability measures (POVM). Density matrix, spectral decomposition, convex decomposition. Bloch sphere and vector. Introduction to quantum entanglement. Quantum correlations, Biorthogonal analysis, Schmidt numbers. Measures of entanglement, Quantum entropy measures. Quantum information. The Schroedinger–HJW theorem. Quantum channels (single qubit, collective channels). Quantum algorithms, Computational and communicational algorithms. The Deutsch–Jozsa algorithm. Quantum teleportation algorithm for states, gates, channels. The LOCC protocol. Quantum walks (QW). Coin-walker Hilbert spaces. The QW channel maps. Quadratic speed ups. Introduction to the Helstrom-Holevo quantum estimation theory. Cramer-Rao bound and quantum Fisher information. Optimal measurements and the symmetric logarithmic derivative operator. Phase estimation problems of quantum states. Temperature estimation and qubit thermometry for closed and open quantum systems.

Χώροι Hilbert καταστατικών διανυσμάτων - Qubit. Θεωρία κβαντικών μετρήσεων, ορθοκανονικές βάσεις, προβολικοί τελεστές. Θετικά τελεστικά μέτρα πιθανότητας POVM. Πίνακας πυκνότητας, φασματική ανάλυση, κυρτοί συνδυασμοί. Σφαίρα κα διάνυσμα Bloch. Εισαγωγή στο κβαντικό εναγκαλισμό. Κβαντικές συσχετίσεις - Διορθογώνια ανάλυση - Αριθμοί Schmidt. Μετρήσεις εναγκαλισμού- Μέτρα κβαντικής εντροπίας. Κβαντική Πληροφορία. Το θεώρημα Schroedinger -HJW. Κβαντικά κανάλια (1-qubit, συλλογικά κανάλια). Κβαντικοί αλγόριθμοι - Αλγόριθμοι υπολογισμών και επικοινωνίας. Ο αλγόριθμος Deutsch--Jozsa. Κβαντική τηλεμεταφορά καταστάσεων, πυλών, καναλιών. Το πρωτόκολο LOCC. Κβαντικοί περίπατοι (QW). Χώροι Hilbert νομίσματος-περιπατητή. Κανάλια τύπου QW. Τετραγωνικές επιταχύνσεις. Εισαγωγή στη θεωρία κβαντικής εκτιμητικής Helstrom-Holevo. Φράγμα Cramér-Rao και κβαντική πληροφορία Fisher. Βέλτιστες μετρήσεις και ο συμμετρικός τελεστής λογαριθμικής παραγώγου. Προβλήματα εκτιμητικής γωνίας φάσης σε κβαντικές καταστάσεις. Εκτιμητική θερμοκρασίας και θερμιδομετρία  qubit για κλειστά και ανοιχτά κβαντικά συστήματα.

 Photo: Jing Liu et al 2020 J. Phys. A: Math. Theor. 53 023001

Διαλέξεις/Lectures

Πέμπτη/Thu,           12:00-13:00  αιθ./Classroom 2041
Παρασκευή/Fri,      12:00-15:00 αιθ.Classroom 2041

Ώρες γραφείου :   Μία ώρα μετά το τέλος των διαλέξεων Πέμπτης και Παρασκευής ή μετά από συνεννόηση

The course will cover tools for optimization problems, where a sequence of (inter-dependent) decisions must be made (dynamically), often under uncertainty about the environment to be optimized (requires learning). Problems like this arise in many modern applications, ranging from autonomous driving and robotics, to game playing (chess, poker, go) and wired/wireless network management. The specific topics covered will be the following: Quick revision of first order optimization methods (gradient descent and stochastic gradient descent); Multi-armed Bandits; First order algorithms for Online Convex Optimization; Online Convex Optimization with instantaneous and long-term constraints; Distributed online convex optimization; backpressure algorithms for Stochastic Network Optimization; Markov Decision Processes; Tabular Reinforcement Learning - RL (Q-Learning, SARSA); Policy gradient methods; Approximate (e.g. Deep) RL solutions such as Deep Q Networks, Actor-Critic methods, and the recent AlphaGo and AlphaZero algorithms, that have been very successful in popular strategic games; Generalized policy rollout for online learning; The class will include both take home exercises, as well as a programming assignment.

Time series data are indexed by the time dimension. This feature imposes a natural order and distinguishes time series from "static" data.  Every data scientist is bound to face at some point in their career time series data.  This course focuses on the fundamental concepts underlying the analysis of time series and some of the methods that can be used to model such data. 

Time series analysis has a long history which led to the development of classical methods such as ARIMA. The machine learning revolution has introduced new tools for time series analysis which are based on deep neural networks (DNNs). However, for most of the everyday problems faced by data scientists, ARIMA and its generalizations perform as well (or even better) than DNN methods which require a very large amount of data in order to achieve their full potential.

This course will focus on classical methodologies of time series analysis.  While the latter are not based on DNNs, they are today considered components of the broad machine learning framework.

SPRING SEMESTER 2023-2024

Tuesday 7-9pm

Wednesday 5-7pm

ROOM 145.Π58

Time series data are indexed by the time dimension. This feature imposes a natural order and distinguishes time series from "static" data.  Every data scientist is bound to face at some point in their career time series data.  This course focuses on the fundamental concepts underlying the analysis of time series and some of the methods that can be used to model such data. 

Time series analysis has a long history which led to the development of classical methods such as ARIMA. The machine learning revolution has introduced new tools for time series analysis which are based on deep neural networks (DNNs). However, for most of the everyday problems faced by data scientists, ARIMA and its generalizations perform as well (or even better) than DNN methods which require a very large amount of data in order to achieve their full potential.

This course will focus on classical methodologies of time series analysis.  While the latter are not based on DNNs, they are today considered components of the broad machine learning framework.

SPRING SEMESTER 2022-2023

Tuesday 5-7pm

Thursday 5-7pm

ROOM 145.Π58