EfficientNet
Overview
EfficientNet is a family of convolutional neural network architectures introduced to address the problem of scaling CNNs in a principled and systematic way. Rather than arbitrarily increasing network depth or width, EfficientNet proposes a compound scaling method that jointly scales depth, width, and input resolution using a fixed set of scaling coefficients.
This approach demonstrated that carefully balanced scaling can yield significantly better accuracy–efficiency tradeoffs than conventional scaling strategies, and it influenced subsequent CNN design, particularly for resource-constrained and production environments.
Architectural Characteristics
- Compound scaling of depth, width, and input resolution
- Use of Mobile Inverted Bottleneck (MBConv) blocks
- Depthwise separable convolutions
- Squeeze-and-Excitation (SE) modules
- Progressive stage-wise scaling across the network
- Family defined by a base model and scaled variants
Design Rationale
Prior CNN architectures typically scaled along a single dimension (e.g., making networks deeper), often leading to diminishing returns or inefficient use of parameters. EfficientNet was designed to demonstrate that model capacity should be increased in a balanced way across multiple dimensions.
The compound scaling rule enforces this balance, ensuring that increases in depth, width, and resolution reinforce each other rather than becoming bottlenecks. The use of MBConv blocks and SE modules further improves parameter efficiency and representational capacity.
Training Paradigm
- Supervised training with cross-entropy loss
- Extensive use of data augmentation and regularization
- Standard gradient-based optimization
- Typically trained on large-scale image datasets
- Scaled variants share architectural structure but differ in capacity and resolution
Notable Variants
- EfficientNet-B0 (base model)
- EfficientNet-B1 through EfficientNet-B7
Benchmark Performance (Reference)
Historical reference results on ImageNet under standard training protocols:
| Model | Dataset | Metric | Result |
|---|---|---|---|
| EfficientNet-B0 | ImageNet | Top-1 Accuracy | ~77% |
| EfficientNet-B4 | ImageNet | Top-1 Accuracy | ~83% |
| EfficientNet-B7 | ImageNet | Top-1 Accuracy | ~84% |