Course Description
The primary aim of this course is to equip students with a comprehensive understanding of the mathematical foundations and practical implementations of deep learning for various modalities like vision, text, reinforcement learning, and multimodal combinations. Deep learning, in the context of this course, pertains to the training and application of neural networks as prediction models for various scenarios.
The course intends to train students from the ground up, starting from Linear Regression models, to advanced architectures similar to Large Language Models/ChatGPT. In that sense, the course provides an extensive description of the different research directions in Deep Learning and includes practical coding aspects, touching upon essential areas such as data loading, training, parameter tuning, and performance evaluation.
This course serves as a good introduction for more advanced AI courses, such as 50.035 Computer Vision and 50.040 Natural Language Processing.
Pre-requisite or Co-requisite
50.007 Machine Learning or 40.319 Statistical & Machine Learning (ESD)
This course necessitates proficiency in Python programming and a basic understanding of calculus, linear algebra, probabilities, and statistics. A pre-existing knowledge of matrix operations, derivative computation for multi-variable functions, and understanding of the gradient descent algorithm will be beneficial.
Learning Objectives
Upon successful completion of this course, students will be able to:
- Grasp the concept of learning from data, generalization, and overfitting.
- Explain the decision boundary of a linear neuron in the context of a general vector space.
- Understand the application of backpropagation.
- Understand the different predictors and suitable loss functions for different prediction problems.
- Understand the fundamentals of convolutional neural networks, and their uses for simple computer vision tasks.
- Explain data augmentation methods.
- Understand the fundamentals of recurrent neural networks, and their uses for sequential data, such as time series or text.
- Gain insights into Natural Language Processing (NLP), transformers, and their role in language understanding and generation.
- Understand the implementation and use of the PyTorch framework for data loading, training, and performance evaluation of deep neural networks.
- Construct custom datasets for images, sequential data, and multi-modal data.
- Understand the concept of adversarial machine learning, and how to make a neural network robust to adversarial attacks.
- Understand the fundamentals of graph convolutional neural networks and how they can be used to process graph datasets.
- Understand the functioning of a generative AI models, such as the generative adversarial neural network models and simple diffusion models.
- Understand and apply advanced concepts like Explainable AI (XAI) and reinforcement learning (RL).
- Understand the functioning and applications of multi-modal neural networks such as LLMs and ChatGPT.
Measurable Outcomes
By the end of this course, students will be able to:
- Evaluate the suitability of deep learning models and loss functions for different prediction task types.
- Compute the terms used in backpropagation for a given neural network topology.
- Differentiate when to use different types of neural network layers (e.g. fully connected layers, convolutional layers, etc.) for a given model.
- Create code for custom datasets for image and text data.
- Create code for neural networks for vision and sequence classification.
- Analyze the degree of overfitting by comparing different pairs of train/test loss curves.
- Implement data augmentation methods and transfer learning in code and evaluate their impact on performance.
- Produce sample code for simple transformers, graph convolutional neural networks, and generative deep neural networks and explain some possible use cases for each.
- Create code for simple Reinforcement Learning applications and see how they relate to Deep Learning.
- Understand how advanced models, such as LLMs and ChatGPT, reuse and combine all the notions introduced in this course.
Topics Covered
This course will cover a range of topics including, but not limited to:
- Introduction to Deep Learning, linking to the contents of the Machine Learning course
- Supervised learning with neural networks
- Understanding the neural networks architecture
- Optimizing and training a neural network
- Convolutional neural networks for Computer Vision
- Recurrent neural networks for sequential data, and Natural Language Programming
- Data Augmentation
- Transfer Learning
- Adversarial Machine Learning on Neural Networks
- Graph Convolutional Neural Networks for graph datasets
- Generative Neural Networks, such as AutoEncoders, Generative Adversarial Networks and Diffusion models
- Explainable AI
- Fundamentals of reinforcement learning and their uses in Deep Learning
Textbook(s) and/or Other Required Material
The following resources will be integral to this course:
- “Deep Learning” by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Available at http://www.deeplearningbook.org/
- “Deep Learning with Python” by François Chollet. Available at https://www.manning.com/books/deep-learning-with-python
- Additional reading materials and videos will be shared during the course duration.