What is computer vision?
Turning pixels into useful signals.

Computer vision is a family of tasks, not one magic camera.

A vision system receives visual data—often a photograph, a video frame, or a sequence of frames—and produces an output defined by its task. It might classify an image, localize objects, mark regions, read text, or connect the visual content to a natural-language question.

The distinction matters. A model trained to classify an entire image as “street” or “forest” may not be able to say where the bicycle is. An object detector can locate likely objects, while a segmentation model marks their pixels. A model that reads a sign does not automatically verify that the sign is current or that its surroundings match a caption.

Before comparing models, describe the output you actually need. A vague goal such as “understand the image” is hard to measure and easy to overclaim.

Different questions require different outputs.

Pixels become a representation the model can use.

An image file is converted into numeric input. A learned model processes patterns in that input and returns the output its task was trained for. Convolutional neural networks became a widely used approach; Vision Transformers showed that a Transformer can also work on sequences of image patches for recognition tasks.

These internal representations are not a literal, human-readable description of everything in the scene. The model’s behavior depends on its architecture, training objective, examples, and the way the application prepares and interprets the input.

For a practical account of training examples and model evaluation, see what machine learning is ↗. For models that connect images with words, read the guide to multimodal AI ↗.

Some models let you point to what they should isolate.

Promptable segmentation is one example of a more interactive vision task. Segment Anything (SAM) accepts prompts such as a point or box and returns a mask for an object or region. The prompt narrows the task; a person still needs to inspect whether the mask follows the intended boundary.

Vision–language models take another route. CLIP, for example, learns representations from paired images and text, allowing image descriptions to be compared with visual inputs. This can support search and classification with language, but similarity is not a guarantee that the model has noticed every relevant detail.

Image analysis and image generation are different directions of work. See how generative image models create pictures ↗, and how to trace an image’s source and context ↗.

A good average score can hide the exact failure you care about.

Lighting, camera position, blur, compression, background, and image quality can change what reaches a model. An evaluation set that does not resemble the real setting can make performance look more dependable than it is. A fluent caption or high confidence number does not show that the model has checked its claim against the world.

1. Define the output. Decide what counts as a correct label, box, mask, extracted phrase, or comparison.
2. Use realistic examples. Include different devices, lighting, viewpoints, backgrounds, and cases where the answer is uncertain.
3. Inspect errors by group. Look for repeated failures across image conditions and affected users, not only one overall score.
4. Separate prediction from action. Add review and escalation before a visual result makes a consequential decision.
5. Review data handling. Images can contain faces, documents, addresses, and other sensitive details; understand where they go and how long they are kept.

For a broader approach to measuring AI systems, read how to read a benchmark ↗ and how to evaluate a system in realistic conditions ↗.

Follow the visual system from input to evidence.

Continue with multimodal AI ↗, image generation ↗, and the AI field guide ↗.