A Decade of Vision Models: From YOLO to SAM — The Evolution of Computer Vision

Forum topic · 小凯 · 2026-02-24

Summary

This forum post chronicles twelve years of computer vision progress, from AlexNet's 2012 breakthrough through YOLO's real-time detection revolution and Meta AI's Segment Anything Model (SAM). It traces object detection milestones including the R-CNN family, YOLOv1 through YOLO11, SSD, RetinaNet, DETR, and RT-DETR; segmentation advances from FCN and U-Net to Mask R-CNN, SAM, and SAM 2; and foundational backbones like ResNet, ViT, Swin Transformer, and ConvNeXt. Key performance figures are included, such as YOLOv1's 45 FPS, YOLOv4's 65.7 mAP on COCO, SAM's 50ms mask generation, and the SA-1B dataset of 11M images and 1.1B masks. The post concludes with trends toward foundation models, multimodal fusion (CLIP, GPT-4V, LLaVA), edge efficiency, and 3D/video understanding.

A forum post on zhichai.net reviews the major milestones in computer vision from 2012 to 2024, covering object detection, segmentation, backbone architectures, and multimodal models.

Chapter 1: The Golden Age of Object Detection (2012–2024)

2012 — AlexNet: Although a classification model, it proved the power of deep CNNs, cutting ImageNet error from 26% to 15% and kickstarting deep learning in vision.
2014–2015 — R-CNN family: R-CNN introduced Selective Search + CNN (47 s/image on GPU); Fast R-CNN added ROI Pooling (0.3 s/image); Faster R-CNN replaced Selective Search with an RPN, becoming the first end-to-end two-stage detector.
2015 — YOLOv1: Framed detection as regression with a single forward pass over a 7×7 grid; 45 FPS (155 FPS for Fast YOLO), 63.4 mAP on VOC 2007; weak on small/dense objects.
2016 — YOLOv2/YOLO9000: Batch Normalization, anchor boxes, multi-scale training, Darknet-19; 9000 classes via WordTree; 40 FPS, 78.6 mAP.
2017 — SSD and RetinaNet: SSD used multi-scale feature maps (46 FPS, 74.3 mAP); RetinaNet's Focal Loss addressed class imbalance and surpassed two-stage accuracy.
2018 — YOLOv3: Darknet-53 backbone, 3-scale prediction, multi-label logistic classifier; 57.9 mAP-50 on COCO at 30 FPS.
2020 — YOLOv4 and YOLOv5: YOLOv4 (CSPDarknet53, SPP, PANet, Mish, Bag of Freebies; 65.7 mAP, 65 FPS); YOLOv5 (Ultralytics PyTorch, engineering-focused, community favorite).
2021–2022 — YOLO proliferation: YOLOX (anchor-free, decoupled head, SimOTA); YOLOv6 (Meituan, RepVGG, quantization-friendly); YOLOv7 (E-ELAN, optimized scaling).
2023 — YOLOv8 and RT-DETR: YOLOv8 added instance segmentation and pose estimation; Baidu's RT-DETR achieved real-time end-to-end detection without NMS.
2024 — YOLOv9/v10/YOLO11: YOLOv9 (PGI, GELAN); YOLOv10 (Tsinghua, NMS-free training, consistent dual assignments); YOLO11 (CNN + Transformer hybrid with attention).

Chapter 2: The Rise of Segmentation (2014–2023)

2014 — FCN: First end-to-end fully convolutional segmentation network with upsampling.
2015 — U-Net: Encoder-decoder with skip connections; de facto standard for medical imaging, 50,000+ citations.
2017 — Mask R-CNN: Added a mask branch to Faster R-CNN with ROI Align.
2020 — DETR: First end-to-end Transformer detector; no anchors or NMS; slow training and weak on small objects.
2023 — SAM (Meta AI, April 2023): Promptable segmentation (points, boxes, masks, text) with zero-shot transfer; ViT image encoder + prompt encoder + lightweight mask decoder; trained via a data engine on SA-1B (11M images, 1.1B masks); ~50 ms masks, ambiguity-aware multi-mask output; viewed as the first vision foundation model, cutting annotation costs by 90%+.
2024 — SAM 2: Extends to video object segmentation with temporal consistency and the SA-V dataset.

Chapter 3: Other Key Milestones

2014 — VGGNet: 16–19 layers, standardized 3×3 convolutions.
2015 — ResNet: Residual connections enabled 152+ layer training; still a backbone of choice.
2016 — DenseNet: Dense connectivity and feature reuse.
2017 — SENet: Squeeze-and-Excitation channel attention, plug-and-play.
2018 — CBAM, ECA: Lightweight channel + spatial attention modules.
2019 — EfficientNet: Compound scaling of depth/width/resolution.
2020 — ViT: Pure Transformer vision model; spawned Swin, PVT, DeiT.
2021 — Swin Transformer: Hierarchical shifted-window attention.
2022 — ConvNeXt: Modernized pure CNN matching ViT performance.
2021–2024 — Multimodal: CLIP (contrastive image-text, zero-shot classification); GPT-4V and Gemini; open-source LLaVA and MiniGPT-4 with visual instruction tuning.

Chapter 4: Trends and Outlook

Architecture timeline: CNN exploration (2012–2014) → depth revolution (2015–2016) → attention (2017–2019) → Transformers (2020–2021) → efficiency and multimodality (2022–2024). Detection paradigm: R-CNN → YOLO → DETR → RT-DETR. Segmentation paradigm: FCN → U-Net → Mask R-CNN → SAM.

Future directions identified:

Foundation models: pretraining + prompting, zero-/few-shot as standard.
Multimodal fusion: vision-language models, unified end-to-end architectures, early world models.
Efficiency and edge: compression, quantization, NAS, edge deployment.
3D and video: NeRF, video understanding, spatiotemporal modeling.

Key Milestone Timeline

2012 AlexNet · 2014 R-CNN, VGG · 2015 ResNet, YOLOv1 · 2016 YOLOv2, SSD · 2017 Mask R-CNN, RetinaNet · 2018 YOLOv3 · 2020 YOLOv4/v5, DETR, ViT · 2021 YOLOX, Swin · 2022 YOLOv6/v7 · 2023 YOLOv8, SAM, RT-DETR · 2024 YOLOv9/v10/v11, SAM 2

References

YOLO paper: https://arxiv.org/abs/1506.02640
SAM paper: https://arxiv.org/abs/2304.02643
Papers With Code: https://paperswithcode.com/