Contributing back into PyTorch

[egpbos]

Following #41, and given our generalization budget, we should think about whether we can contribute parts of this project back into PyTorch.

Related: #89, because maybe our audio submodule could be contributed back, or maybe be used to provide bugfixes.

Other possibilities we discussed were the video handling tools we are now building and the way we handle multi-modal input. It is not hugely likely that video will be generalizable, since there is already a torchvision package which has video tools. For multi-modal models, there is a big package from Facebook called MMF, but it is hard to see whether we can and should contribute there, so that should be looked into.

Any other ideas we can discuss here.