Use Capsule Nets

Capsule nets (CapsNet) are a recent class of neural nets aiming to provide an alternative to classic convolutional neural networks in computer vision. CapsNets are not yet used in real-world applications, but are a promising topic of research. This example demonstrates the original Capsule Network trained on the MNIST classification task and explores the properties of the capsule vectors it computes.

Obtain a pre-trained CapsNet model from the Wolfram Neural Net Repository.

CapsNet are not pure classifiers; they are also an autoencoder and can therefore reconstruct their input data.

Instead of having scalar neurons holding final scores for each class, a CapsNet produces capsule vectors, which encode the score in their norm. In addition to the scalar case, information about the features of the detected instance are encoded in the orientation of capsule vectors, in a similar way to feature vectors. Extract the length-16 capsule vector with highest norm from the "Pick" block of the network.

The "Reconstruct" block is trained to reconstruct the input image from the information contained in the capsule vector, acting as a regularizer. Successful reconstruction proves that capsule vectors encode the characteristics of the specific instance detected in their orientation. Obtain the reconstructor.

Feed the previously obtained capsule vector to the reconstructor.

The space spanned by capsule vectors possesses features reminding latent spaces of classic and variational autoencoders. Experiment with changing the capsule vector. Changing the capsule vector can result in variations of features such as thickness or can entirely transform the instance into another class.

Interpolate between capsule vectors of different digits.