Computer Vision Questions

Short answer / decision
For a 5k-image classification task I would pick a CNN (or a CNN backbone with transfer learning). CNNs’ strong locality and translation equivariance give better sample efficiency and lower compute for small datasets.

Comparison

Inductive biases

• CNNs: Local connectivity, weight sharing, translation equivariance — built-in priors for images.
• Transformers: Minimal image-specific bias; rely on attention to learn relationships.

Data requirements

• CNNs: Work well with limited data, especially with transfer learning and augmentation.
• Transformers: Tend to need much larger datasets (or large pretraining) to match CNNs.

Compute behavior

• CNNs: Convolution scales linearly with image size; efficient on GPU, lower memory for similar accuracy.
• Transformers: Self-attention is O(n^2) in patch tokens; higher memory and compute, especially at high resolution.

Receptive field

• CNNs: Local early, grows with depth; easy to control with dilation/residuals.
• Transformers: Global from first layer via attention; better long-range modeling but learned, not inductive.

Typical performance

• Small datasets (few k images): CNNs outperform or are more reliable via transfer learning.
• Very large datasets / pretraining (millions): Vision transformers (ViT + hybrids) can match or exceed CNNs.

Why CNN for 5k images

• High sample efficiency, mature pretrained models (ResNet, EfficientNet), lower compute and easier regularization. Practical approach: start with a pretrained CNN, heavy augmentation, fine-tune, consider semi-supervised or few-shot methods if needed.