CS6910 / CS7015 · Deep Learning · IIT Madras

Lecture notes for Deep Learning, lectures 2–14

Notes for Prof. Mitesh M. Khapra's course, one page per lecture, built from the official slides and the lecture videos. Each page has diagrams of the key ideas, the maths written out, timestamp links into the module videos, and questions to test yourself.

Start with Lecture 2 Course page

Lectures

The lectures and topics follow the Schedule on the course page. Each lecture is a set of module videos (M1, M2, …); the notes link into the right module at the right moment.

LectureTopicsSlidesNotes
Foundations
02McCulloch–Pitts neurons, perceptrons and MLPsMcCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPsSlidesReady
03Sigmoid neurons, gradient descent, feedforward networksSigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural NetworksSlidesReady
04Feedforward networks and backpropagationFeedforward Neural Networks, BackpropagationSlidesReady
Training deep networks
05Optimization: from gradient descent to AdamGradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, AdamSlides 1 · Slides 2Ready
06RegularizationBias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, DropoutSlidesReady
07Pre-training, activations, initialization, batch normGreedy Layerwise Pre-training, Better activation functions, Better weight initialization methods, Batch NormalizationSlidesReady
Convolutional networks
08Convolutional neural networksConvolutional Neural Networks, LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet, ResNetSlidesReady
09Visualizing and fooling CNNsVisualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural NetworksSlidesReady
Sequences and language
10Word representationsLearning Vectorial Representations Of WordsSlidesReady
11Recurrent neural networks and BPTTRecurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTTSlidesReady
12GRUs and LSTMsGated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanishing gradient problem with LSTMsSlidesReady
13Encoder–decoder models and attentionEncoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical AttentionSlidesReady
14TransformersTransformers: Multi-headed Self Attention, Cross AttentionSlidesReady

Using these notes

  • Every section starts with a timestamp chip, such as M3 · 12:40, that opens that module's video at that moment.
  • The slides are the backbone of each page; the videos fill in the explanations. Each page links to its official slides.
  • Lectures 8–13 have NPTEL's human-made captions. Lectures 2–7 and 14 only have YouTube's auto-generated captions, so those notes rely more on the slides for exact wording and formulas. Each page's footer says which it used.
  • Charts built on synthetic data say so in their caption. They illustrate the idea, not a dataset from the course.
  • Lecture 1 (history of deep learning) is not covered.