Generative and Self-Supervised ML

The students that finish this course will know the basic principles of unsupervised deep learning methods and will be able to use them in simple applications.

Day 1 (Monday January 19th)

Session 1 /10:00-12:30 (10` break in between at 11am)

1. Introduction to unsupervised deep learning
2. Self supervised learning
3. Generative deep learning: Autoregressive models
4. Practice with Colab

Day 2 (Tuesday January 20th)

Session 1 /10:00-12:30 (10` break in between at 11am)

1. Generative deep learning: Autoregressive models (cont)
2. Generative deep learning: Flow models
3. Practice with Colab

Day 3 (Wednesday January 21st)

Session 1 /10:00-12:30 (10` break in between at 11am)

1. Generative deep learning: Latent Variables Models, VAEs
2. Generative deep learning: Implicit models, GANs
3. Practice with Colab

Day 4 (Thursday January 22nd)

Session 1 /10:00-12:30 (10` break in between at 11am)

1. Generative deep learning: Implicit models, GANs
2. Generative deep learning: Diffusion/Flow matching models
3. Practice with Colab

Day 5 (Friday January 23rd)

Session 1 /10:00-12:30 (10` break in between at 11am)

1. Generative deep learning: Diffusion/Flow matching models

Practice with Colab

• Javier Béjar Alonso – HPAI Group – Visitor researcher
•  Pablo Martín Torres – HPAI Group – Research engineer

Javier Béjar Alonso is associate professor at Universitat Politècnica de Catalunya and visitor researcher at BSC. He currently teaches at the Facultat d’informàtica de Barcelona in the computer science bachelor degree and the artificial intelligence bachelor and master degrees. He works on artificial intelligence and machine learning with an emphasis on applications to medicine and engineering. Currently his research is focused on the development of deep learning generative models for the generation of synthetic multimodal and time series datasets for medical applications.

Pablo Martín Torres is a mathematician and AI engineer. He has a background on Stochastic Geometry research at the Freie Universität Berlin. At present, Pablo is working as an AI research engineer at the Barcelona Supercomputing Center, focussing on multimodal flow matching and diffusion models, explainability and machine unlearning.

The objectives of this course is to understand the basic principles of unsupervised deep learning methods. The course is organized around two paradigms self supervised learning and generative models.

The part of the course about self supervised learning will explain the basics of models for learning representations from data using neural network based on stablished auto supervision methods, covering reconstruction based methods, common sense tasks based method and contrastive and non contrastive learning methods

The part of the couse about generative models will explain the basics of the five paradigms of generative neural networks. This includes autoregressive models, models based on normalizing flows, models based on latent variables (variational auto encoders), implicit models (generative adversarial networks), and models based on denoising and score matching (Denoising diffusion models/flow matching models).

<p>Basic knowledge of python programming and torch library, use of jupyter notebooks/google colab, basic knowlege of probability estimation and neural networks</p>

<p>Each student should bring his/her own laptop</p>