Limitations and user implementations
There are a number of limitations and allowed parameter combinations within the multi-view-AE
framework. These restrictions are set in the multiviewae/base/validation.py
file and will need to be updated should the user wish to add their own implementations.
Allowed parameter types are also set in the multiviewae/base/validation.py
file.
Limitations and allowed parameter combinations
Distribution classes
It should be noted that currently the multivariate normal class, multiviewae.base.distributions.MultivariateNormal
, implements a multivariate gaussian with a diagonal covariance matrix.
Further work will involve implementing a multivariate normal class where the off-diagonal elements of the covariance matrix can be specified or learnt.
Encoder distribution
The multiviewae.base.distributions.Default
class must be used for the vanilla autoencoder and adversarial autoencoder implementations where no distribution is specified.
Either the multiviewae.base.distributions.Normal
or multiviewae.base.distributions.MultivariateNormal
classes must be used for variational models.
For adversarial autoencoders with gaussian posterior, i.e. gaussian encoding distributions, the multiviewae.base.distributions.Normal
or multiviewae.base.distributions.MultivariateNormal
classes can be used
if their are coupled with a variational encoder architecture, e.g. multiviewae.architectures.mlp.VariationalEncoder
.
Encoder and prior distribution combinations
Currently, the encoder distribution must be the same as the prior distribution.
Models which support CNN architectures
Many of the autoencoder models in the multi-view-AE
support CNN encoder and decoder network architectures. The JMVAE
and MMVAE
models do not currently support these architectures.
This will be addressed in further work.
Adding user designed classes
Users are able to implement their own network architectures, datamodules, datasets and distributions. This should cover the majority of classes the user should wish to edit and offers lots of flexibility when implementing a model. For any other classes, users must have access to the source code and the class must be added to the supported classes in the
multiviewae/base/validation.py
file.
User designed network architectures
User designed MLP network classes must be implemented in a mlp.py
file and named one of; Encoder
, VariationalEncoder
, Decoder
, and VariationalDecoder
depending on the network type.
CNN network classes must be implemented in a cnn.py
file and named one of; Encoder
, VariationalEncoder
, and Decoder
depending on the network type.
Networks must except and return the same parameters as the respective multi-view-AE
counterpart.
For example, variational encoder networks must return mu
and logvar
in the form of a Torch.tensor
.
Please see the Architectures section for information on input and output parameters of encoder and decoder networks.
Implemented classes must have a forward
method.
User designed datamodules and datasets
User designed datamodules must implement a setup
method, a train_dataloader
method and a val_dataloader
method and must accept the same parameters as the multi-view-AE
counterpart. The datamodule class name must end in DataModule
.
User designed datasets must implement a __getitem__
method and must accept the same parameters as the multi-view-AE
counterpart. The dataset class name must end in Dataset
.
NOTE User implemented datasets must also provide a is_path_ds
parameter to indicate whether the input data is a path to the data for the data to be loaded when the __getitem__
method is called or whether the data is stored in memory.
User designed distributions
User designed distributions must implement a log_likelihood
and _sample
method.