What does the TensorFlows operation conv2d_transpose

What does TensorFlows `conv2d_transpose ()` operation do?

conv2d_transpose () simply transposes the weights and tilts them 180 degrees. Then the standard conv2d () applies. "Transposing" practically means that the order of the "columns" in the weight tensor is changed. Please check the example below.

Here is an example that uses folds with Stride = 1 and Padding = 'SAME'. It's a simple case, but the same considerations could be applied to the other cases.

Let's say we have:

  • Input: MNIST image of 28x28x1, form = [28,28,1]
  • Convolution layer: 32 filters of 7 x 7, weighting form = [7, 7, 1, 32], name = W_conv1

When we do the convolution of the input, the activations of the shape have the form: [1,28,28,32].

Where from:

To get the "deconvolution" or "transposed convolution" we can use conv2d_transpose () for the convolution activations as follows:

OR using conv2d () we need to transpose and flip the weights:

Here we change the order of the "columns" from [0,1,2,3] to [0,1,3,2]. So from [7, 7, 1, 32] a tensor with the form = is obtained [7,7,32,1]. Then we flip the weights:

Then we can compute the convolution with conv2d () as:

And we will get the same result as before. The same result can also be obtained with conv2d_backprop_input ():

The results are shown here:

Test of conv2d (), conv2d_tranposed () and conv2d_backprop_input ()

We can see that the results are the same. To see it better please check my code below:

https://github.com/simo23/conv2d_transpose

Here I am replicating the output of the conv2d_transpose () function using the conv2d () standard.