ANN Deep Learning Architectures

What is Deep Learning (DL)?

Deep Learning is part of a broader family of Machine Learning methods, which is based on Artificial Neural Networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. Deep learning is a technique that teaches computers to do what comes naturally to humans: learn by example. It is a field that is based on learning and improving on its own by examining computer algorithms. While machine learning uses simpler concepts, Deep Learning works with Artificial Neural Networks, which are designed to imitate how humans think and learn. Deep learning structures algorithms in layers to create an “artificial neural network” that can learn and make intelligent decisions on its own.

The word "deep" in "deep learning" refers to the number of layers through which the data is transformed. More precisely, deep learning systems have a substantial Credit Assignment Path (CAP) depth. The CAP is the chain of transformations from input to output. CAPs describe potentially causal connections between input and output.

For a Feed-Forward Neural Network, the depth of the CAPs is that of the network and is the number of hidden layers plus one (as the output layer is also parameterized).

For Recurrent Neural Networks (RNNs), in which a signal may propagate through a layer more than once, the CAP depth is potentially unlimited. No universally agreed-upon threshold of depth divides shallow learning from deep learning, but most researchers agree that deep learning involves CAP depth higher than 2.

Deep learning is a key technology behind driverless cars, enabling them to recognize a stop sign, or to distinguish a pedestrian from a lamppost.

Which are the best machine learning or deep learning algorithm or architectures?

There is no single "best" machine learning or deep learning algorithm or architecture that works for all tasks, as the choice depends on (a) nature of the problem, (b) the available data, and (c) the desired performance. However, some popular and widely-used algorithms and architectures have proven effective for a variety of tasks.