卷积层大小计算公式

Given:

• our input layer has a width of W and a height of H
• our convolutional layer has a filter size F
• we have a stride of S
• a padding of P
• and the number of filters K,

the following formula gives us the width of the next layer: W_out =[ (W−F+2P)/S] + 1.

The output height would be H_out = [(H-F+2P)/S] + 1.

And the output depth would be equal to the number of filters D_out = K.

The output volume would be W_out * H_out * D_out.

Being able to calculate the number of parameters in a neural network is useful since we want to have control over how much memory a neural network uses.

Setup

H = height, W = width, D = depth

• We have an input of shape 32x32x3 (HxWxD)
• 20 filters of shape 8x8x3 (HxWxD)
• A stride of 2 for both the height and width (S)
• Zero padding of size 1 (P)

Output Layer

• 14x14x20 (HxWxD)

Solution

There are 756560 total parameters. That's a HUGE amount! Here's how we calculate it:

(8 * 8 * 3 + 1) * (14 * 14 * 20) = 756560

8 * 8 * 3 is the number of weights, we add 1 for the bias. Remember, each weight is assigned to every single part of the output (14 * 14 * 20). So we multiply these two numbers together and we get the final answer.

Now we'd like you to calculate the number of parameters in the convolutional layer, if every neuron in the output layer shares its parameters with every other neuron in its same channel.

This is the number of parameters actually used in a convolution layer (tf.nn.conv2d()).

Setup

H = height, W = width, D = depth

• We have an input of shape 32x32x3 (HxWxD)
• 20 filters of shape 8x8x3 (HxWxD)
• A stride of 2 for both the height and width (S)
• Zero padding of size 1 (P)

Output Layer

• 14x14x20 (HxWxD)

Hint

With parameter sharing, each neuron in an output channel shares its weights with every other neuron in that channel. So the number of parameters is equal to the number of neurons in the filter, plus a bias neuron, all multiplied by the number of channels in the output layer.

Solution

There are 3860 total parameters. That's 196 times fewer parameters! Here's how the answer is calculated:

(8 * 8 * 3 + 1) * 20 = 3840 + 20 = 3860

That's 3840 weights and 20 biases. This should look similar to the answer from the previous quiz. The difference being it's just 20 instead of (14 * 14 * 20). Remember, with weight sharing we use the same filter for an entire depth slice. Because of this we can get rid of 14 * 14 and be left with only 20.