What is ReLU?

ReLU (rectified linear unit) is an activation function that keeps positive values as they are and turns negative values into zero. It is simple and fast to compute, and it reduces the vanishing gradient problem, which makes it very suitable for deep neural networks; this is why it is widely used in applications such as computer vision and deep learning in general.