The goal of the loss functions are to drive automated learnings by being a differentiable function that has a meaningful derivative.

Metrics that humans might use to measure the performance of an AI system might not be suitable because they might have big flat sections and large jumps.

We want the loss function to react to small changes in the inputs so we can keep improving the weights of the neural network.

Thus, loss functions will have to compromise measuring the real goal and being a function that can be optimized using its gradient.

mlai