Part of course:

Paper Summary: Generative Adversarial Nets

- Introduction
- Adversarial Net
- Experiments
- Strengths
- Weakness
- Possible Extensions

NaN.

Paper Summary: Generative Adversarial Nets

- The paper proposes an adversarial approach for estimating generative models where one model (generative model) tries to learn a data distribution and another model (discriminative model) tries to distinguish between samples from the generative model and original data distribution.
- Link to the paper

- Two models - Generative Model(
*G*) and Discriminative Model(*D*) - Both are multi-layer perceptrons.
*G*takes as input a noise variable*z*and outputs data sample*x(=G(z))*.*D*takes as input a data sample*x*and predicts whether it came from true data or from*G*.*G*tries to minimise*log(1-D(G(z)))*while*D*tries to maximise the probability of correct classification.- Think of it as a minimax game between 2 players and the global optimum would be when
*G*generates perfect samples and*D*can not distinguish between the samples (thereby always returning 0.5 as the probability of sample coming from true data). - Alternate between
*k*steps of training*D*and 1 step of training*G*so that*D*is maintained near its optimal solution. - When starting training, the loss
*log(1-D(G(z)))*would saturate as*G*would be weak. Instead maximise*log(D(G(z)))* - The paper contains the theoretical proof for global optimum of the minimax game.

- Datasets: MNIST, Toronto Face Database, CIFAR-10
- Generator model uses RELU and sigmoid activations.
- Discriminator model uses maxout and dropout.
- Evaluation Metric: Fit Gaussian Parzen window to samples obtained from
*G*and compare log-likelihood.

- Computational advantages
- Backprop is sufficient for training with no need for Markov chains or performing inference.
- A variety of functions can be used in the model.
- Since
*G*is trained only using the gradients from*D*, fewer chances of directly copying features from the true data. - Can represent sharp (even degenerate) distributions.

*D*must be well synchronised with*G*.- While
*G*may learn to sample data points that are indistinguishable from true data, no explicit representation can be obtained.

- Conditional generative models.
- Inference network to predict
*z*given*x*. - Implement a stochastic extension of the deterministic Multi-Prediction Deep Boltzmann Machines
- Using discriminator net or inference net for feature selection.
- Accelerating training by ensuring better coordination between
*G*and*D*or by determining better distributions to sample*z*from during training.

Read more…(375 words)

Mark as completed

Part of lists:

Previous

Paper Summary: DRAW: A Recurrent Neural Network For Image Generation

Next

Paper Summary: Human-Level Control Through Deep Reinforcement Learning

About the author:

Shagun Sodhani

Loading…

Have a question? Ask here…

Post

Part of course:

Paper Summary: Generative Adversarial Nets

- Introduction
- Adversarial Net
- Experiments
- Strengths
- Weakness
- Possible Extensions

About the author

Shagun Sodhani

Ready to join our community?

Sign up below to automatically get notified of new courses, get **reminders** to finish ones you subscribe to, and **bookmark** lessons to read later.

Continue with Facebook

— OR —

Your Full Name

Email address

I have an account. Log in instead

By signing up, you agree to our Terms and our Privacy Policy.

Popular Courses

New Courses

Get in touch

Copyright 2016-18, Compose Labs Inc. All rights reserved.