#concept-pamphlet #class #todo go through the entire CS229 lecture notes and UPDATE my definitions of various terms. please ππ» important to do before school starts
Linear regression
What is the notation for a training set?
Incudes training example with input and output features
?
Note that the superscript β(i)β is simply an index / i-th item in the training set and has nothing to do with exponentiation.
?
?
What are the benefits of squaring something like in the cost function of the ordinary least squares regression model?
?
- The result of the square is always positive
- Squaring puts more weight on larger errors/differences
- The result is always differentiable
- The result corresponds to the assumption of normally distributed errors
What is the least-squares cost function J?
?
What is learning rate, in the context of training neural nets? In gradient descent?
?
A hyperparameter that determines the size of steps taken during gradient descent. A higher learning rate might converge faster but overshoot the minimum, while a lower rate converges more slowly but reliably
In the context of gradient descent, it is Ξ±:
What is the equation for gradient descent?
?
where J is the cost function and Ξ± is learning rate.
Or this for a single training example:
What is the difference between stochastic gradient descent and batch gradient descent?
What is the likelihood function?
What is the log likelihood function?
What is the maximum likelihood function?
i donβt understand: LMS update rule / Widrow-Hoff learning rule and how that was derived
Classification and logistic regression
perceptron learning algorithm
Newtonβs method - for finding a zero of a function
hessian
fisher scoring
Generalized linear models
A class of distributions is in the exponential family if it can be written in the form:
