NEURAL NETWORKS ZERO TO HERO - LECTURE 1 - BUILDING MICROGRAD

Video Lecture	The spelled-out intro to neural networks and backpropagation: building micrograd (youtube.com)
Code Implementation	nn_zero_to_hero/micrograd_from_scratch.ipynb at main · AndresNamm/nn_zero_to_hero (github.com)
Extra Video explaining when a parameter is updated with Gradient	NN_ZERO_TO_HERO complement - YouTube

Summary

Building Micrograd - a small neural network that explains the important Math concepts behind neural networks
Builds out an interesting neural network where each parameter, number in the neural network is a Python class with +/-/*/: operators and derivatives. This allows to simplify each derivative operation into operation between 2 variables.

Derivative example
Gradient calculation
Chain rule of derivatives
- $\frac{dy}{dx} = f’(g(x))g’(x)$
Different functions and their derivatives
- Sum
- Division
- Tanh
Forward Pass
Loss function
Backpropagation
- Gradient descent
- Chain rule
- How often a variable is updated in Gradient descent. Additional explaining video