We went through key papers during 2 months on Generative models with Generative models study group. In order to share our insights and knowledge obtained, we provide honest subjective opinion on those papers and generally about limitations and strengths of so striking popular generative models! Thanks for study group participants for preparing this overview.
- Overview of the Plug and Play Generative Networks: http://www.aihelsinki.com/gans-overview-plug-and-play/
- WaveNet: http://www.aihelsinki.com/generative-models-wavenet/
NIPS 2016 Adversarial Networks Tutorial
The purpose of the study group was to familiarize ourselves into a new are called Generative Adversarial Networks.
The method was to go through some the latest papers on the topic and discuss those among the group. The starting point and a level setter for the team was a NIPS 2016 tutorial by a well known researches in this area Dr. Ian Goodfellow (https://arxiv.org/abs/1701.00160).
There is much excitement in this area and one of the legends of this field, Yann LeCun, labeled Adversarial Training as one of the most promising areas in Deep Learning (https://www.quora.com/session/Yann-LeCun/1).
The maturity of GANs, therefore, is both a challenge and a source for excitement. A challenge since the theory and practise are still fairly new and according to the papers we studied the stability of GANs are still very much an unsolved problem. This means that papers but especially implementations of GANs are still raw. Several implementations we studied had “magic numbers” that just made the system stable but the reasoning why that is is still missing. Some implementations of GANs, especially in the field of image recognition have proved to be very useful and some applications have already been developed that show great potential. GANs seem to offer a new way of handling and composing images using “arithmetic” in the generative space.
These findings were indeed very exciting but the immaturity of the field made the papers fairly challenging. The background information that was needed sometimes felt a bit overwhelming.
Despite of the challenges the study group was always very supportive and people were ready to help and advice in the tedious process. All the presentations I attended were well prepared and challenges and findings were nicely presented. I would strongly recommend these groups to anyone interested in machine learning.