English | MP4 | AVC 1280×720 | AAC 44KHz 2ch | 265 lectures (57h 17m) | 21.9 GB
Master deep learning in PyTorch using an experimental scientific approach, with lots of examples and practice problems.
Deep learning is increasingly dominating technology and has major implications for society.
From self-driving cars to medical diagnoses, from face recognition to deep fakes, and from language translation to music generation, deep learning is spreading like wildfire throughout all areas of modern technology.
But deep learning is not only about super-fancy, cutting-edge, highly sophisticated applications. Deep learning is increasingly becoming a standard tool in machine-learning, data science, and statistics. Deep learning is used by small startups for data mining and dimension reduction, by governments for detecting tax evasion, and by scientists for detecting patterns in their research data.
Deep learning is now used in most areas of technology, business, and entertainment. And it’s becoming more important every year.
How does deep learning work?
Deep learning is built on a really simple principle: Take a super-simple algorithm (weighted sum and nonlinearity), and repeat it many many times until the result is an incredibly complex and sophisticated learned representation of the data.
Is it really that simple? mmm OK, it’s actually a tiny bit more complicated than that but that’s the core idea, and everything else — literally everything else in deep learning — is just clever ways of putting together these fundamental building blocks. That doesn’t mean the deep neural networks are trivial to understand: there are important architectural differences between feedforward networks, convolutional networks, and recurrent networks.
Given the diversity of deep learning model designs, parameters, and applications, you can only learn deep learning — I mean, really learn deep learning, not just have superficial knowledge from a youtube video — by having an experienced teacher guide you through the math, implementations, and reasoning. And of course, you need to have lots of hands-on examples and practice problems to work through. Deep learning is basically just applied math, and, as everyone knows, math is not a spectator sport!
What is this course all about?
Simply put: The purpose of this course is to provide a deep-dive into deep learning. You will gain flexible, fundamental, and lasting expertise on deep learning. You will have a deep understanding of the fundamental concepts in deep learning, so that you will be able to learn new topics and trends that emerge in the future.
Please note: This is not a course for someone who wants a quick overview of deep learning with a few solved examples. Instead, this course is designed for people who really want to understand how and why deep learning works; when and how to select metaparameters like optimizers, normalizations, and learning rates; how to evaluate the performance of deep neural network models; and how to modify and adapt existing models to solve new problems.
You can learn everything about deep learning in this course.
In this course, you will learn
- Theory: Why are deep learning models built the way they are?
- Math: What are the formulas and mechanisms of deep learning?
- Implementation: How are deep learning models actually constructed in Python (using the PyTorch library)?
- Intuition: Why is this or that metaparameter the right choice? How to interpret the effects of regularization? etc.
- Python: If you’re completely new to Python, go through the 8+ hour coding tutorial appendix. If you’re already a knowledgeable coder,
- then you’ll still learn some new tricks and code optimizations.
- Google-colab: Colab is an amazing online tool for running Python code, simulations, and heavy computations using Google’s cloud services. No need to install anything on your computer.
Unique aspects of this course
- Clear and comprehensible explanations of concepts in deep learning.
- Several distinct explanations of the same ideas, which is a proven technique for learning.
- Visualizations using graphs, numbers, and spaces that provide intuition of artificial neural networks.
- LOTS of exercises, projects, code-challenges, suggestions for exploring the code. You learn best by doing it yourself!
- Active Q&A forum where you can ask questions, get feedback, and contribute to the community.
- 8+ hour Python tutorial. That means you don’t need to master Python before enrolling in this course.
What you’ll learn
- The theory and math underlying deep learning
- How to build artificial neural networks
- Architectures of feedforward and convolutional networks
- Building models in PyTorch
- The calculus and code of gradient descent
- Fine-tuning deep network models
- Learn Python from scratch (no prior coding experience necessary)
- How and why autoencoders work
- How to use transfer learning
- Improving model performance using regularization
- Optimizing weight initializations
- Understand image convolution using predefined and learned kernels
- Whether deep learning models are understandable or mysterious black-boxes!
- Using GPUs for deep learning (much faster than CPUs!)
Table of Contents
Introduction
1 How to learn from this course
2 Using Udemy like a pro
Download all course materials
3 Downloading and using the code
4 My policy on code-sharing
Concepts in deep learning
5 What is an artificial neural network
6 How models learn
7 The role of DL in science and knowledge
8 Running experiments to understand DL
9 Are artificial neurons like biological neurons
About the Python tutorial
10 Should you watch the Python tutorial
Math, numpy, PyTorch
11 PyTorch or TensorFlow
12 Introduction to this section
13 Spectral theories in mathematics
14 Terms and datatypes in math and computers
15 Converting reality to numbers
16 Vector and matrix transpose
17 OMG it’s the dot product!
18 Matrix multiplication
19 Softmax
20 Logarithms
21 Entropy and cross-entropy
22 Minmax and argminargmax
23 Mean and variance
24 Random sampling and sampling variability
25 Reproducible randomness via seeding
26 The t-test
27 Derivatives intuition and polynomials
28 Derivatives find minima
29 Derivatives product and chain rules
Gradient descent
30 Overview of gradient descent
31 What about local minima
32 Gradient descent in 1D
33 CodeChallenge unfortunate starting value
34 Gradient descent in 2D
35 CodeChallenge 2D gradient ascent
36 Parametric experiments on g.d
37 CodeChallenge fixed vs. dynamic learning rate
38 Vanishing and exploding gradients
39 Tangent Notebook revision history
ANNs (Artificial Neural Networks)
40 The perceptron and ANN architecture
41 A geometric view of ANNs
42 ANN math part 1 (forward prop)
43 ANN math part 2 (errors, loss, cost)
44 ANN math part 3 (backprop)
45 ANN for regression
46 CodeChallenge manipulate regression slopes
47 ANN for classifying qwerties
48 Learning rates comparison
49 Multilayer ANN
50 Linear solutions to linear problems
51 Why multilayer linear models don’t exist
52 Multi-output ANN (iris dataset)
53 CodeChallenge more qwerties!
54 Comparing the number of hidden units
55 Depth vs. breadth number of parameters
56 Defining models using sequential vs. class
57 Model depth vs. breadth
58 CodeChallenge convert sequential to class
59 Diversity of ANN visual representations
60 Reflection Are DL models understandable yet
Overfitting and cross-validation
61 What is overfitting and is it as bad as they say
62 Cross-validation
63 Generalization
64 Cross-validation — manual separation
65 Cross-validation — scikitlearn
66 Cross-validation — DataLoader
67 Splitting data into train, devset, test
68 Cross-validation on regression
Regularization
69 Regularization Concept and methods
70 train() and eval() modes
71 Dropout regularization
72 Dropout regularization in practice
73 Dropout example 2
74 Weight regularization (L1L2) math
75 L2 regularization in practice
76 L1 regularization in practice
77 Training in mini-batches
78 Batch training in action
79 The importance of equal batch sizes
80 CodeChallenge Effects of mini-batch size
Metaparameters (activations, optimizers)
81 What are metaparameters
82 The wine quality dataset
83 CodeChallenge Minibatch size in the wine dataset
84 Data normalization
85 The importance of data normalization
86 Batch normalization
87 Batch normalization in practice
88 CodeChallenge Batch-normalize the qwerties
89 Activation functions
90 Activation functions in PyTorch
91 Activation functions comparison
92 CodeChallenge Compare relu variants
93 CodeChallenge Predict sugar
94 Loss functions
95 Loss functions in PyTorch
96 More practice with multioutput ANNs
97 Optimizers (minibatch, momentum)
98 SGD with momentum
99 Optimizers (RMSprop, Adam)
100 Optimizers comparison
101 CodeChallenge Optimizers and… something
102 CodeChallenge Adam with L2 regularization
103 Learning rate decay
104 How to pick the right metaparameters
FFNs (Feed-Forward Networks)
105 What are fully-connected and feedforward networks
106 The MNIST dataset
107 FFN to classify digits
108 CodeChallenge Binarized MNIST images
109 CodeChallenge Data normalization
110 Distributions of weights pre- and post-learning
111 CodeChallenge MNIST and breadth vs. depth
112 CodeChallenge Optimizers and MNIST
113 Scrambled MNIST
114 Shifted MNIST
115 CodeChallenge The mystery of the missing 7
116 Universal approximation theorem
More on data
117 Anatomy of a torch dataset and dataloader
118 Data size and network size
119 CodeChallenge unbalanced data
120 What to do about unbalanced designs
121 Data oversampling in MNIST
122 Data noise augmentation (with devset+test)
123 Data feature augmentation
124 Getting data into colab
125 Save and load trained models
126 Save the best-performing model
127 Where to find online datasets
Measuring model performance
128 Two perspectives of the world
129 Accuracy, precision, recall, F1
130 APRF in code
131 APRF example 1 wine quality
132 APRF example 2 MNIST
133 CodeChallenge MNIST with unequal groups
134 Computation time
135 Better performance in test than train
FFN milestone projects
136 Project 1 A gratuitously complex adding machine
137 Project 1 My solution
138 Project 2 Predicting heart disease
139 Project 2 My solution
140 Project 3 FFN for missing data interpolation
141 Project 3 My solution
Weight inits and investigations
142 Explanation of weight matrix sizes
143 A surprising demo of weight initializations
144 Theory Why and how to initialize weights
145 CodeChallenge Weight variance inits
146 Xavier and Kaiming initializations
147 CodeChallenge Xavier vs. Kaiming
148 CodeChallenge Identically random weights
149 Freezing weights during learning
150 Learning-related changes in weights
151 Use default inits or apply your own
Autoencoders
152 What are autoencoders and what do they do
153 Denoising MNIST
154 CodeChallenge How many units
155 AEs for occlusion
156 The latent code of MNIST
157 Autoencoder with tied weights
Running models on a GPU
158 What is a GPU and why use it
159 Implementation
160 CodeChallenge Run an experiment on the GPU
Convolution and transformations
161 Convolution concepts
162 Feature maps and convolution kernels
163 Convolution in code
164 Convolution parameters (stride, padding)
165 The Conv2 class in PyTorch
166 CodeChallenge Choose the parameters
167 Transpose convolution
168 Maxmean pooling
169 Pooling in PyTorch
170 To pool or to stride
171 Image transforms
172 Creating and using custom DataLoaders
Understand and design CNNs
173 The canonical CNN architecture
174 CNN to classify MNIST digits
175 CNN on shifted MNIST
176 Classify Gaussian blurs
177 Examine feature map activations
178 CodeChallenge Softcode internal parameters
179 CodeChallenge How wide the FC
180 Do autoencoders clean Gaussians
181 CodeChallenge AEs and occluded Gaussians
182 CodeChallenge Custom loss functions
183 Discover the Gaussian parameters
184 The EMNIST dataset (letter recognition)
185 Dropout in CNNs
186 CodeChallenge How low can you go
187 CodeChallenge Varying number of channels
188 So many possibilities! How to create a CNN
CNN milestone projects
189 Project 1 Import and classify CIFAR10
190 Project 1 My solution
191 Project 2 CIFAR-autoencoder
192 Project 3 FMNIST
193 Project 4 Psychometric functions in CNNs
Transfer learning
194 Transfer learning What, why, and when
195 Transfer learning MNIST – FMNIST
196 CodeChallenge letters to numbers
197 Famous CNN architectures
198 Transfer learning with ResNet-18
199 CodeChallenge VGG-16
200 Pretraining with autoencoders
201 CIFAR10 with autoencoder-pretrained model
Style transfer
202 What is style transfer and how does it work
203 The Gram matrix (feature activation covariance)
204 The style transfer algorithm
205 Transferring the screaming bathtub
206 CodeChallenge Style transfer with AlexNet
Generative adversarial networks
207 GAN What, why, and how
208 Linear GAN with MNIST
209 CodeChallenge Linear GAN with FMNIST
210 CNN GAN with Gaussians
211 CodeChallenge Gaussians with fewer layers
212 CNN GAN with FMNIST
213 CodeChallenge CNN GAN with CIFAR
RNNs (Recurrent Neural Networks) (and GRULSTM)
214 Leveraging sequences in deep learning
215 How RNNs work
216 The RNN class in PyTorch
217 Predicting alternating sequences
218 CodeChallenge sine wave extrapolation
219 More on RNNs Hidden states, embeddings
220 GRU and LSTM
221 The LSTM and GRU classes
222 Lorem ipsum
Ethics of deep learning
223 Will AI save us or destroy us
224 Example case studies
225 Some other possible ethical scenarios
226 Will deep learning take our jobs
227 Accountability and making ethical AI
Where to go from here
228 How to learn topic X in deep learning
229 How to read academic DL papers
Python intro Data types
230 How to learn from the Python tutorial
231 Variables
232 Math and printing
233 Lists (1 of 2)
234 Lists (2 of 2)
235 Tuples
236 Booleans
237 Dictionaries
Python intro Indexing, slicing
238 Indexing
239 Slicing
Python intro Functions
240 Inputs and outputs
241 Python libraries (numpy)
242 Python libraries (pandas)
243 Getting help on functions
244 Creating functions
245 Global and local variable scopes
246 Copies and referents of variables
247 Classes and object-oriented programming
Python intro Flow control
248 If-else statements
249 If-else statements, part 2
250 For loops
251 Enumerate and zip
252 Continue
253 Initializing variables
254 Single-line loops (list comprehension)
255 while loops
256 Broadcasting in numpy
257 Function error checking and handling
Python intro Text and plots
258 Printing and string interpolation
259 Plotting dots and lines
260 Subplot geometry
261 Making the graphs look nicer
262 Seaborn
263 Images
264 Export plots in low and high resolution
Bonus section
265 Bonus content
Resolve the captcha to access the links!