- Neural network is a highly flexible math function
- Stochastic Gradient Descent (SGD) is how we can improve the weights on the NN to improve performance
- Architecture: functional form of the model
- Parameters: Weights
- Predictions: the result. extracted from the data (the input, without label) (data is the independent variable)
- Performance is measured in loss
- Loss depends on the predictions and correct labels
- Labels may also be called targets/dependent variables
A good loss is not necessarily a good metric:
In other words, a good choice for lossis a choice that is easy for stochastic gradient descent to use. But a metric is definedfor human consumption, so a good metric is one that is easy for you to understand - Deep Learning for Coders, Chapter 1