Seeing the World: An Introduction to Convolutional Neural Networks (CNNs) | FromZero2AI Blog

Images are high-dimensional structures. If we feed a small 256 \times 256 color image into a standard Fully Connected (dense) neural network, the input layer must handle 256 \times 256 \times 3 = 196,608 features. If the first hidden layer has just 512 neurons, the layer would require over 100 million parameters! This leads to severe overfitting and parameter explosion.

Furthermore, dense networks fail to capture spatial local dependencies—for instance, a cat's eye is still an eye regardless of where it appears in the image. Convolutional Neural Networks (CNNs) solve this by utilizing localized filters to capture patterns like edges, textures, and shapes.

Core Components of a CNN

A standard CNN architecture consists of three main building blocks repeated in sequence: Convolutional Layers, Pooling Layers, and Fully Connected (Dense) Layers.

Convolutional Neural Networks: 3D architectural diagram showing inputs, conv feature maps, pooling, and dense output layers Convolutional Neural Networks: 3D architectural diagram showing inputs, conv feature maps, pooling, and dense output layers

1. The Convolutional Layer

At the heart of the CNN is the convolution operation. A convolutional layer consists of a set of learnable kernels (or filters). A kernel is a small matrix (typically 3 \times 3 or 5 \times 5) that slides across the input image.

The Mathematics of 2D Convolution

At each position, we calculate the element-wise multiplication of the kernel values and the overlapping input image pixels, then sum them up:

(I * K)(i, j) = \sum_{m} \sum_{n} I(i - m, j - n) K(m, n)

Where:

I is the input image.
K is the kernel.
(i, j) is the index of the output pixel.

This sliding operation generates a 2D grid called a Feature Map (or Activation Map).

Hyperparameters: Kernel Size, Stride, and Padding

To configure a convolutional layer, we must define three key hyperparameters:

Kernel Size (F): The spatial dimensions of the filter (e.g., 3 for a 3 \times 3 filter).
Stride (S): The number of pixels the filter moves at each step. A stride of 1 moves the filter one pixel at a time; a stride of 2 moves it two pixels, halving the output size.
Padding (P): Adding dummy border pixels (usually zeros) around the input image.
- Valid Padding: No padding (P=0). The output size shrinks.
- Same Padding: Padding is adjusted so the output spatial dimensions match the input dimensions.

Output Size Formula

Given an input size of W_{in}, filter size F, padding P, and stride S, the output size W_{out} is calculated as:

W_{out} = \left\lfloor \frac{W_{in} - F + 2P}{S} \right\rfloor + 1

2. Pooling Layers

Pooling layers reduce the spatial dimensions of the feature maps, which minimizes the number of parameters and computation in the network. Pooling also helps the network achieve translation invariance (making it robust to small shifts in the input).

Max Pooling: Slides a window across the feature map and selects the maximum value. This is the most common technique as it extracts the most dominant features (e.g. sharp edges).
Average Pooling: Computes the average value in the window.

For a 2 \times 2 Max Pooling layer with a stride of 2, the spatial dimension of the feature map is cut exactly in half, while the depth (number of channels) remains unchanged.

3. PyTorch CNN Implementation

Let's build a simple CNN class in PyTorch designed for a classic image classification dataset like CIFAR-10 (32 \times 32 images with 3 color channels).

import torch
import torch.nn as nn
import torch.nn.functional as F

class SimpleCNN(nn.Module):
    def __init__(self, num_classes=10):
        super(SimpleCNN, self).__init__()
        
        # Conv Layer 1: Input channels = 3 (RGB), Output channels = 16, Kernel = 3x3
        # Padding = 1 keeps the dimensions same
        self.conv1 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, padding=1)
        
        # Conv Layer 2: Input channels = 16, Output channels = 32, Kernel = 3x3
        self.conv2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, padding=1)
        
        # Max Pooling Layer: Window = 2x2, Stride = 2
        self.pool = nn.MaxPool2d(kernel_size=2, stride=2)
        
        # Fully Connected Layers
        # After two pooling operations, the spatial size of a 32x32 image is:
        # 32x32 -> (Pool 1) -> 16x16 -> (Pool 2) -> 8x8
        # Flattened size: 32 channels * 8 * 8 pixels = 2048 features
        self.fc1 = nn.Linear(32 * 8 * 8, 128)
        self.fc2 = nn.Linear(128, num_classes)
        
        # Batch Normalization for stability
        self.bn1 = nn.BatchNorm2d(16)
        self.bn2 = nn.BatchNorm2d(32)

    def forward(self, x):
        # First Block: Conv -> Batch Normalization -> ReLU -> Pool
        x = self.pool(F.relu(self.bn1(self.conv1(x))))
        
        # Second Block: Conv -> Batch Normalization -> ReLU -> Pool
        x = self.pool(F.relu(self.bn2(self.conv2(x))))
        
        # Flatten the feature maps for the fully connected layers
        x = x.view(-1, 32 * 8 * 8)
        
        # Fully connected layers with dropout to prevent overfitting
        x = F.relu(self.fc1(x))
        x = F.dropout(x, p=0.5, training=self.training)
        x = self.fc2(x)
        
        return x

# Test forward pass with dummy image batch: Batch size = 4, Channels = 3, Height = 32, Width = 32
dummy_input = torch.randn(4, 3, 32, 32)
model = SimpleCNN()
outputs = model(dummy_input)
print("Output tensor shape:", outputs.shape)  # Expected: [4, 10]

Summary

The SimpleCNN above is a teaching model, and it is worth being clear about what it will and will not do. Trained on CIFAR-10 it lands somewhere in the 70-75% accuracy range — respectable for a network you can read in one screen, and nowhere near what you would ship. Almost nobody trains an architecture like this from scratch for a real task. The standard move is to take a ResNet or EfficientNet pretrained on ImageNet, replace the final classification layer, and fine-tune. With a few thousand labeled images that approach will beat a hand-built CNN trained on the same data by a wide margin, because the early layers have already learned edge and texture detectors from a million images you do not have.

The most common bug when writing your own is the flatten. The line x = x.view(-1, 32 * 8 * 8) hardcodes an assumption that the input was 32×32 and passed through exactly two 2×2 pooling layers. Feed it a 64×64 image and it either errors or, worse, silently reshapes into a wrong batch size. Adding a third pooling layer breaks it the same way. Use nn.AdaptiveAvgPool2d((1, 1)) or x.flatten(1) with a nn.LazyLinear head and the dimension arithmetic stops being your problem — recomputing that number by hand every time you change the architecture is a reliable source of wasted afternoons.

One conceptual caveat on pooling: the translation invariance it provides is weaker than the usual description implies. Max pooling gives robustness to shifts of a pixel or two, not to a cat appearing in a different corner of the frame. That larger invariance comes from the depth of the stack and from data augmentation — random crops and flips during training — far more than from the pooling layers themselves. If your model fails on off-center objects, augmentation is the lever, not a bigger pooling window.