Brain Dump

Machine Learning


Updated at: 12 Mar 2019 23:37:44


ANN builds pyrimidal cells in the cerebral cortex. Uses shortcuts to jump over layers.

Motivation for skipping over layers is to avoid problem of vanishing gradients by reusing activations from a previous layer until adjacent layer has learnt weights.

The weights adapt to mute the upstream layer, amplify the previously skipped layer.

Forward propagation for a simple RNN: alttext