Computer vision is the branch of artificial intelligence that teaches machines to pull meaning from images and video, the same way a person glances at a chest X-ray or a busy intersection and understands what’s happening. As of July 17, 2026, it’s doing that work at commercial scale. Waymo passed 500,000 paid driverless rides per week earlier this year. The FDA has cleared more than 1,300 AI-enabled medical devices, and nearly 80% of them read radiology images. This guide walks through how the technology works right now, what’s actually deployed, and where it still falls short.
Quick answer: what is computer vision?
Computer vision is a field of AI focused on extracting information from digital images and video. Modern systems use deep learning models, mostly convolutional neural networks (CNNs), vision transformers (ViTs), or hybrids of the two, to do tasks like object recognition, segmentation, classification, and tracking. It powers medical imaging, self-driving cars, facial recognition, factory inspection, cashierless retail, and traffic management, among many others.
From Hough transforms to vision transformers: a short history
The field has been around longer than most people assume. In the 1980s, the Hough Transform let machines detect lines and circles in a picture. That was state of the art for years. Handcrafted feature detectors dominated until 2012.
Then AlexNet happened. A convolutional neural network trained on ImageNet posted results that were, at the time, close to super-human on image recognition. CNNs ran the field for the next decade. Every serious vision system, from Tesla’s early autopilot to Google Photos search, was built on the same basic idea: stack convolution layers, learn local features from edges up to whole objects, and let the hierarchy do the work.
The next shift came in 2020. Researchers borrowed the transformer architecture from language models, chopped images into fixed-size patches, and treated each patch like a word. That’s the Vision Transformer, or ViT. Instead of building up local features, ViTs look at every patch in relation to every other patch from the first layer onward. Six years later, that idea has reshaped the whole field.
How does machine vision actually work?
Curious what AI could do for your business?
No jargon and no hard sell. Just a friendly look at where AI fits, and where it doesn't.
At the lowest level, a computer vision model takes a grid of pixel values and produces a structured output: a label, a bounding box, a segmentation mask, a set of keypoints, or a description. The steps are roughly:
- Input processing: Resize, normalise, and often augment the image (flips, crops, colour jitter) so the model doesn’t overfit.
- Feature extraction: A CNN slides small filters across the image to detect edges, textures, and shapes. A ViT splits the image into patches and embeds each one as a vector.
- Reasoning: Deeper layers combine those features. In a CNN this happens hierarchically. In a ViT it happens through self-attention, where every patch weighs its relationship to every other patch.
- Output head: A final layer converts the internal representation into whatever the task needs. Classification produces a label. Detection produces boxes. Segmentation produces a per-pixel mask.
- Training: The whole thing is trained on labelled examples, updating millions or billions of parameters until predictions match ground truth.
The interesting part is what these two families of models are good at, and where they break.
CNNs vs vision transformers: which one wins?
Both work. The trade-offs are real, and picking the wrong one for your problem is a common and expensive mistake.
CNNs have a built-in spatial bias. They assume that nearby pixels are related and that a cat is still a cat when it moves across the frame. That bias, called locality and translation equivariance, is why CNNs generalise well from smaller datasets. They also scale linearly with image resolution, which matters when you’re running inference on a phone.
ViTs make no such assumption. They have to learn spatial structure from data, which is why they’re notoriously data hungry, often needing 100 million or more images to hit their stride. But once they get there, they see globally. A ViT can relate a shadow in the top left of a frame to an object in the bottom right in a single layer. CNNs need many stacked layers to do the same thing.
| Feature | CNNs | Vision Transformers |
|---|---|---|
| Feature learning | Local, hierarchical | Global, all patches at once |
| Inductive bias | Strong spatial bias | Weak, learned from data |
| Data efficiency | Works with smaller datasets | Needs massive datasets |
| Compute scaling | Linear with resolution | Quadratic with patches |
| Interpretability | Easier (Grad-CAM, filter viz) | Harder (attention maps) |
| Best fit | Edge devices, limited data | Cloud compute, big data, fine detail |
On the numbers, ViTs are ahead. ViT-L/16 and Swin Transformer-Base both clear 87.8% Top-1 accuracy on ImageNet, edging out the best CNN variants by 0.2 to 0.3 percentage points. In tasks needing fine-grained discrimination, like radiology and pathology, the gap widens. Kaya and colleagues found in 2025 that ViTs also hold up better against distance and occlusion in face recognition, sometimes matching the fastest CNNs on inference speed.
But there’s a catch, and it matters for anyone actually shipping software.
Why edge deployment still favours CNNs (mostly)
A ViT with 196 patches needs about 150MB per forward pass. A ResNet CNN needs around 50MB for the same job. Most phones cap out at 4GB of RAM shared across every app. That’s the wall.
Three techniques get around it:
- Pruning removes low-attention heads and unimportant connections. Model size drops sharply with only a small accuracy hit.
- Quantisation cuts weight precision from 32-bit floats down to 8-bit integers. Memory shrinks, inference speeds up on integer hardware.
- Knowledge distillation trains a small “student” model to imitate a larger “teacher”. The student is deployable. The teacher stays on a server.
Here’s the interesting part. Compression works better on ViTs than on CNNs, according to the Edge AI and Vision Alliance. So even though ViTs start out too heavy for edge hardware, they compress harder without falling apart. The gap is closing.
The rise of hybrid models
Pure CNNs and pure ViTs are both losing ground to hybrids that mix the two. The pitch: keep the CNN’s local feature extraction, add the transformer’s global attention, get the best of both.
ConvNeXt, released by Meta in 2022, went the other way. It rebuilt the CNN using Transformer design principles: larger kernels, fewer activations, LayerNorm. By 2026 it lands within 0.3% of ViT accuracy while keeping CNN-friendly deployment characteristics. For production systems with real latency budgets, it’s often the pragmatic pick.
CoAtNet stacks CNN blocks before Transformer blocks. It beats pure ViT on ImageNet by 1% with 30% fewer FLOPs. In medical imaging, TransUNet wedges a 12-layer transformer between a CNN encoder and decoder, which lets it hold onto tissue-level detail while still reasoning globally about an organ.
“Hybrid architectures merge convolutional layers’ local feature extraction with transformers’ global self-attention to capture both fine details and overall context.” — Emergent Mind, 2026
If you’re picking an architecture for a new project in 2026, the honest answer is: start with a hybrid. Pure ViT if you have web-scale data and cloud GPUs. Pure CNN if you’re deploying to a $30 microcontroller. Hybrid for almost everything in between.
Computer vision in healthcare: real gains, real bias
Medical imaging is where computer vision has moved fastest into daily practice. The U.S. computer vision in healthcare market is projected to grow from $3.93 billion in 2024 to $11.53 billion by 2029. Globally, revenue is expected to top $33.4 billion by 2036.
Specific wins:
- Mammography: A real-world German study of 463,094 women found AI-supported mammography raised breast cancer detection by 17.6%.
- Stroke care: Hospitals using tools like Brainomix 360 Stroke doubled their thrombectomy rates, from 2.3% to 4.6%.
- Pathology: Vision transformers now match or slightly beat state-of-the-art CNNs on tumour detection in whole-slide images, according to a 2022 comparative study by Deininger and colleagues.
Now the harder part. These systems inherit the biases of their training data, and the training data has gaps that show up in patient outcomes.
A 2021 review of 21 public skin-cancer image datasets, 106,950 images in total, found that only 2,436 recorded skin tone at all. Of those, exactly one was the darkest skin type. A model can’t learn what it never sees. The bias carries into deployed tools, too. A Radiology study of an FDA-cleared mammography AI found false-positive scores were 50% more likely for Black women than for white women. A Nature analysis in 2025 documented CNNs underdiagnosing disease on chest X-rays for female, Black, Hispanic, and low-income patients. The Kaiser Family Foundation reviewed the literature and found a significant association between AI use in health care and the exacerbation of racial disparities.
Here’s what most articles about medical AI leave out: even a technically accurate model fails if it doesn’t slot into the radiologist’s workflow. If an AI output requires two extra clicks inside a PACS system, adoption craters. I’ve watched hospital pilots die not because the model was wrong, but because loading its results took eleven seconds instead of two. Data labelling has the same friction. Annotating a scan takes a specialist away from a patient. Every label is a diagnosis delayed.
Computer vision in autonomous vehicles: cameras vs lidar
Waymo runs a full sensor stack: cameras, radar, and LiDAR, fused together. Tesla runs vision-only. In 2026 they’re both on public roads with real passengers, and the debate over which approach is safer is still live.
The camera-only case is straightforward. Cameras are cheap. They see colour, lane markings, and traffic signs. They mimic what humans use. The problem: they struggle with glare, low light, and estimating depth from a flat image.
The sensor-fusion case is also straightforward. LiDAR gives centimetre-level 3D depth regardless of lighting. Radar reads velocity directly. Cameras add texture and colour. Any one sensor’s blind spot is covered by another. LiDAR is expensive, does poorly in heavy rain, snow, and fog, and can’t read the colour of a traffic light. So it needs the camera too.
The industry consensus, at least among safety researchers, is that Level 3 and Level 4 autonomy safely requires sensor fusion. No single sensor is enough for every condition on every road.
Where things get legally interesting is liability. When a Level 3 system is driving and it crashes, the manufacturer is on the hook, not the driver. Mercedes-Benz accepts that responsibility whenever its Drive Pilot system is active. BMW briefly did too, then pulled its Level 3 Personal Pilot from the 7 Series in 2026 after weak demand. Most other OEMs won’t take on the liability at all, which is one reason Level 3 rollouts have crawled while Level 2 driver assists have spread everywhere.
Where else computer vision is doing real work in 2026
A partial tour, because listing every application would fill a book:
What could a custom AI agent take off your plate?
We build production-grade AI systems that quietly handle the busywork, so your team can focus on the work that actually matters.
- Manufacturing: Automated inspection of biopharma production lines catches contamination, checks vial integrity, and verifies fill levels. Auto plants use computer vision for exterior and underbody inspection.
- Retail: Cashierless checkout, real-time inventory tracking, and parking-lot occupancy monitoring.
- Agriculture: Crop disease detection and plant health monitoring. Watch out for background bias here. CNNs trained on the popular PlantVillage dataset sometimes learn the background instead of the disease. ViTs handle this better because they reason globally.
- Public infrastructure: Municipal traffic systems analyse CCTV feeds to count vehicles, spot incidents, and retime traffic signals in real time.
- Facial recognition: Now a $11.62 billion market. The access-control segment alone is growing at a 10.5% CAGR through 2033. GDPR-style regulation is shaping deployment, especially in the EU, and the industry is pushing toward multimodal biometrics, on-device processing, and clearer consent flows.
Computer vision vs machine learning: what’s the difference?
They’re not the same thing, and the distinction matters if you’re hiring or budgeting.
Machine learning is the broader field. It’s any system that learns patterns from data instead of being explicitly programmed. Computer vision is one application of machine learning, specifically for image and video data. Most modern computer vision uses deep learning, a subset of machine learning based on neural networks with many layers.
So: all modern computer vision is machine learning. Not all machine learning is computer vision. A fraud detection model reading transaction data is machine learning but not computer vision. A tumour detector reading a CT scan is both.
What to watch next
If you’re deciding whether to build with computer vision in the next 12 months, three things matter more than architecture choice.
First, know your deployment target before you pick a model. Edge and cloud have completely different constraints, and switching later is expensive. Second, audit your training data for representation gaps before you audit your model for accuracy. The bias problem in healthcare AI is a data problem masquerading as a model problem. Third, watch for hybrid architectures continuing to eat both pure CNN and pure ViT market share, especially ConvNeXt-style modernised CNNs, which give you transformer accuracy with CNN operational characteristics. That’s the trajectory that’s actually shipping in production right now.
