Machine Learning for Time Series (Master MVA)


Teaching material and outline of the course Machine Learning for Time Series (Master MVA) during 2023-2024.

Course description

In many application contexts (health, economy, advertising...), the data collected takes the form of time series. The fundamental challenge then consists in choosing an adapted representation, allowing to take into account the temporal information as well as possible.

Machine Learning for time series gathers a large number of unsupervised or supervised tasks such as prediction, classification, completion/interpolation, query by content/indexation, clustering, segmentation/change-point detection or anomaly detection. But in reality, most of work for a data scientist dealing with temporal data consists in a series of hidden tasks such as:

This course aims to provide an overview of ML techniques to study time series, in different tasks such as pattern extraction and recognition, anomaly detection, prediction, interpolation etc. The course will mostly focus on these often poorly-documented hidden tasks and introduce several recent ML methods that will help the future data scientist to mine, but above all to understand time series. The course will be widely illustrated on real data and problems from current challenges and will emphasize aspects related to data understanding and interpretation. Note that in its current form, the course will only marginally discuss Deep Learning algorithms.

Outline and planning

Lectures will take place on Thursday afternoons at ENS Paris Saclay. Lectures will be on-site (ENS Paris Saclay) and will NOT be filmed or recorded. Lectures will be in French but all material (slides, homeworks...) is in English. For the tutorial sessions two options will be available: Thursday mornings will be remote on Zoom and Thursday afternoons will be onsite at ENS Paris Saclay. Attendance to the lectures and tutorial sessions is mandatory.

05/10/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Introduction
 pdf
Lecture 1: Pattern Recognition and Detection
  1. Problem statement
  2. Comparing time series
    1. Euclidean distance
    2. Normalized Euclidean distance
    3. Dynamic Time Warping (DTW)
  3. Detecting patterns in time series
    1. Euclidean distance
    2. DTW
  4. Learning patterns from time series
    1. Distance-based pattern extraction
    2. Dictionary-based pattern extraction
 pdf
12/10/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Lecture 2: Feature Extraction and Selection
  1. Problem statement
  2. Feature extraction
    1. Stationarity and ergodicity
    2. Statistical features
    3. Spectral features
    4. Local symbolic features
    5. Information theory features
    6. Deep Learning features
    7. Other features
  3. Feature selection
    1. Unsupervised setting
    2. Supervised setting
 pdf
19/10/2023
09:00 → 12:00
Zoom
OR
19/10/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Tutorial 1 on Lectures 1 & 2
 pdf
26/10/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Lecture 3: Models and Representation Learning
  1. Problem statement
  2. Standard models
    1. Sinusoidal model
    2. Trend+Seasonality model
    3. AR models (and variants)
    4. Hidden Markov model
  3. Representation learning
    1. Standard representations
    2. Notion of sparsity
    3. Sparse coding
    4. Dictionary learning
 pdf
09/11/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Lecture 4: Data Enhancement and Preprocessings
  1. Problem statement
  2. Denoising
    1. Filtering
    2. Sparse approximations
    3. Low-rank approximations
    4. Other techniques
  3. Detrending
    1. Least-Square regression
    2. Other techniques
  4. Interpolation of missing samples
    1. Polynomial interpolation
    2. Low-rank interpolation
    3. Model-based interpolation
  5. Outlier removal
    1. Isolated samples
    2. Contiguous samples
 pdf
16/11/2023
09:00 → 12:00
Zoom
OR
16/11/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Tutorial 2 on Lectures 3 & 4
 pdf
23/11/2023
14:00 → 17:00
1Z14
Lecture 5: Change-Point and Anomaly Detection
  1. Problem statement
  2. Change point detection
    1. Dealing with non-stationary time series
    2. Problem statement
    3. Cost functions
    4. Search methods
    5. Finding the number of change points
  3. Anomaly detection
    1. Outlier detection
    2. Statistical methods
    3. Model-based methods
    4. Distance-based methods
  4. Evaluation of event detection methods
 pdf
30/11/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Lecture 6: Multivariate Time Series
  1. Problem statement
  2. Models for multivariate time series
    1. Vector autoregressive models
    2. Multivariate dictionary learning
  3. Graph signal processing
    1. Concepts and definitions
    2. Graph Fourier Transform (GFT)
    3. Bandlimitedness and smoothness
    4. Graph filtering
    5. Graph learning
 pdf
07/12/2023
09:00 → 12:00
Zoom
OR
07/12/2023
14:00 → 17:00
Amphi Hodgkins (0I10)
Tutorial 3 on Lectures 5 & 6
 pdf
20/12/2023 (all day)
22/12/2023 (afternoon)
11/01/2024 (all day)
12/01/2024 (all day)
Zoom
Oral presentations

Registration and mailing list

A registration form will be sent to all MVA students to subscribe to the course mailing-list. The final registration date is set to October 15th 2023 : no registration will be allowed after this date.

Tutorials

Tutorials will consist in interactive sessions where the students will be introduced to useful Python packages for time series analysis and have the opportunity to use and apply the different algorithms studied during the lectures. After each tutorial, students will be asked to work in pairs on a small project that will consist in (almost) direct applications of the algorithms seen in the tutorial sessions. Students are required to bring their own personal computer during the tutorial sessions : details on installation and required configuration will be provided before the first tutorial session. Attendance at at least one of the two tutorials sessions (on-site or remote) is mandatory: absences must be justified, otherwise you will receive a FAIL. Missed or late assignments will also give you a FAIL in the course.

Validation

Additional ressources

Useful references
 pdf
List of possible topics/projects pdf