This post provides summary of the paper by Berthelot et al. 2017. They proposed a robust architecture for GAN with usual training procedure. In order to have stable convergence, they propose use to use equilibrium concept between Generator and Discriminator. The results are much imporoved in terms of both image diversity and visual quality.

paper : Berthelot et al. 2017

Proposed Method

Wasserstein distance for auto-encoders

To study the effects of matching the distribution of errors, an auto-encoder loss is approximated as normal distribution and further Wasserstein distance is computed between auto-encoder loss of real and generated samples.

Pixel-wise autoencoder training loss is given as where is autoencoder function, is target norm, is a sample of dimension .

Using Central limit theorem, if number of pixels are assumed to be large and per pixel loss to i.i.d. the image wise losses approximately follow normal distribution.

Wasserstein distance

Given two normal distributions and with mean and covariances , the distance is given as

when , above equation reduces to


the problem is simplified as

In this paper the aim is to reduce Wasserstein distance between loss distribution and not sample distribution.

GAN Objective

be the normal distibution of the loss where are real samples

be the normal distribution of the loss where is the generator function and are random uniform samples

The possible solutions of maximizing eq(5)



the choice for objective in the paper is (a)

Overall GAN objective is given as


Usually, and are not well balanced and wins easily. To maintain a balance between generator and descriminator. The equilibrium condition is given as :

However, in general settings this is not the case and hence a new parameter is considered.


BEGAN with Equilibrium


both the objectives of generator and discriminator are trained simultaneously


For Generator

img —> conv(3,3) —> {[conv33, conv33] —>upsample(2,2) } x 3—> conv3,3 x 2 —> fc —> emb(h)

For discriminator

emb(h) —> {conv33, conv33, subsample(2,2)} x 3 —> {conv3,3 , conv3,3} -> conv3,3(img)

Leave a Comment