Capsule Networks (CapsNets) – Tutorial

by Super User, 3 months ago
0 0
CapsNets are a hot new architecture for neural networks, invented by Geoffrey Hinton, one of the godfathers of deep learning.

NIPS 2017 Paper:
* Dynamic Routing Between Capsules,
* by Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton

The 2011 paper:
* Transforming Autoencoders
* by Geoffrey E. Hinton, Alex Krizhevsky and Sida D. Wang

CapsNet implementations:
* Keras w/ TensorFlow backend:
* TensorFlow:
* PyTorch:

Hands-On Machine with Scikit-Learn and TensorFlow
O'Reilly, 2017



* At 15:47, in the margin loss equation, the max should be squared, but not the norm: L_k = T_k max(0, m+ − ||v_k||)² + λ (1 − T_k) max(0, ||v_k|| − m−)². Therefore, at 16:08, the network should output a vector whose length (not squared length) is longer than 0.9 for digits that are present, or smaller than 0.1 for digits that are absent. I'll clarify this point in my next video on implementing Capsule Networks.