(i) Residual Network (ResNet)

Residual Network (ResNet)

Residual Network (ResNet) is a Deep Learning Model used for computer vision applications. It is a Convolutional Neural Network (CNN) architecture designed to support hundreds or thousands of convolutional layers. ResNet is primarily designed for image classification and computer vision tasks. It was introduced by He et al. in their 2015 paper "Deep Residual Learning for Image Recognition." The key innovation in ResNet is the introduction of residual connections, also known as skip connections, which allow the network to learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. These connections help mitigate the vanishing gradient problem, allowing the network to be trained efficiently even when it is very deep. ResNets have been widely adopted in various computer vision tasks, such as object detection, image segmentation, and facial recognition.

Previous CNN architectures were not able to scale to a large number of layers, which resulted in limited performance. However, when adding more layers, researchers faced the “vanishing gradient” problem.

Neural networks are trained through a backpropagation process that relies on gradient descent, shifting down the loss function and finding the weights that minimize it. If there are too many layers, repeated multiplications will eventually reduce the gradient until it “disappears”, and performance saturates or deteriorates with each layer added.

ResNet provides an innovative solution to the vanishing gradient problem, known as “skip connections”. ResNet stacks multiple identity mappings (convolutional layers that do nothing at first), skips those layers, and reuses the activations of the previous layer. Skipping speeds up initial training by compressing the network into fewer layers.

In summary, ResNet is a deep convolutional neural network designed for computer vision tasks, with a focus on efficiently training very deep networks using residual connections.