Updated at: 13 Mar 2019 14:37:44


ANN builds pyrimidal cells in the cerebral cortex. Uses shortcuts to jump over layers.

Motivation for skipping over layers is to avoid problem of vanishing gradients by reusing activations from a previous layer until adjacent layer has learnt weights.

The weights adapt to mute the upstream layer, amplify the previously skipped layer.

Forward propagation for a simple RNN: alttext