GAN producing strangely tiled, patternistic results not in line with the output shape

I was developing a GAN to generate 48x48 images of faces. However, the generator makes strange images no matter how much training is done, and no matter how much the discriminator thinks it's fake.

---

untrained output

After 25 epochs

You can see that the outputs are suprisingly related. In both cases the image is made up of 16 "clusters" of small squares. Each cluster is also made up of 16 squares. Each cluster is the same arrangement of 2x2 pixels with a 1 pixel margin, and each has varying opacity.

I expected after 25 epochs, at least the generator would be producing vague shapes of a human head/face. But no matter how many epochs I trained, I would get the same sort of pattern with different squares.

It also seems to be related to the kernel size of the deconv layers.

This is an image from a 5x5 kernel size

At first, I thought it might be a form of mode collapse, but the discriminator was very certain that the generated images were fake.

>>> discriminator(generated_image)


<tf.Tensor: shape=(1, 1), dtype=float32, numpy=array([[5.560048e-11]], dtype=float32)> # 0 is fake

Models:

generator = keras.Sequential([
    keras.layers.Dense(4*4*32, input_shape=(100,), use_bias=False),
    keras.layers.LeakyReLU(),
    
    keras.layers.Reshape((4, 4, 32)),
    
    SpectralNormalization(keras.layers.Conv2DTranspose(64, (2, 2), strides=(2, 2), padding="same", use_bias=False)),
    keras.layers.LeakyReLU(),
    
    SpectralNormalization(keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding="same", use_bias=False)),
    keras.layers.LeakyReLU(),
    
    SpectralNormalization(keras.layers.Conv2DTranspose(3, (2, 2), strides=(3, 3), padding="same", use_bias=False)),
    keras.layers.LeakyReLU(),
    
    SpectralNormalization(keras.layers.Conv2DTranspose(3, (2, 2), strides=(2, 2), padding="same", use_bias=False)),

    
    
    
])


discriminator = keras.Sequential([
    keras.layers.Conv2D(128, (2, 2), strides=(2, 2), input_shape=(96, 96, 3), padding="same"),
    keras.layers.LeakyReLU(),
    
    
    SpectralNormalization(keras.layers.Conv2D(64, (2, 2), strides=(2, 2), padding="same")),
    keras.layers.LeakyReLU(),
    
    SpectralNormalization(keras.layers.Conv2D(32, (2, 2), strides=(2, 2), padding="same")),
    keras.layers.LeakyReLU(),
    
    SpectralNormalization(keras.layers.Conv2D(16, (2, 2), strides=(2, 2), padding="same")),
    keras.layers.LeakyReLU(),
    
    SpectralNormalization(keras.layers.Conv2D(8, (2, 2), strides=(2, 2), padding="same")),
    keras.layers.LeakyReLU(),
    

    
    keras.layers.Flatten(),
    keras.layers.Dense(1, activation="sigmoid")
    
])

The training loop was taken from the Tensorflow GAN tutorial

cross_entropy = tf.keras.losses.BinaryCrossentropy()

def discriminator_loss(real_output, fake_output):
    real_loss = cross_entropy(tf.ones_like(real_output), real_output)
    fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)
    total_loss = real_loss + fake_loss
    return total_loss

def generator_loss(fake_output):
    return cross_entropy(tf.ones_like(fake_output), fake_output)

generator_optimizer = tf.keras.optimizers.Adam(1e-4)
discriminator_optimizer = tf.keras.optimizers.Adam(1e-4)

@tf.function
def train_step(images):
    noise = tf.random.normal([32, 100])

    with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:
        generated_images = generator(noise, training=True)

        real_output = discriminator(images, training=True)
        fake_output = discriminator(generated_images, training=True)

        gen_loss = generator_loss(fake_output)
        disc_loss = discriminator_loss(real_output, fake_output)

    gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)
    gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)

    generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
    discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))

from GAN producing strangely tiled, patternistic results not in line with the output shape