Reviewed and updated October 3, 2026.
Object detection is a computer-vision task that identifies what objects are present in an image or video frame and predicts where each object is located. Most detectors return a class label, a bounding box, and a confidence score for every reported object.
Unlike image classification, which often assigns one or more labels to an entire image, object detection can find several objects and locate each one separately. That difference makes detection useful whenever a system needs to answer both “what is here?” and “where is it?”
What an object detector produces
Imagine a photograph of a park path containing a person, a dog, a bicycle, and a bench. A detector might report four predictions:
- person — bounding box around the walker — confidence 0.96
- dog — bounding box around the animal — confidence 0.94
- bicycle — bounding box around the bike — confidence 0.91
- bench — bounding box around the seat — confidence 0.88
Each prediction combines three ideas: classification names the object, localization describes its position, and the confidence score indicates how strongly the model supports that candidate prediction. Post-processing may remove or combine overlapping candidates, depending on the model and deployment pipeline.
Object detection vs classification vs segmentation
On narrow screens, scroll the comparison table sideways. Keyboard: focus the table region and use the left and right arrow keys.
| Computer-vision task | Question it answers | Main output | Example |
|---|---|---|---|
| Classification | What is in the image? | Image-level category label or labels | “This image contains a dog.” |
| Object detection | What objects are present, and where? | Object labels, locations, and confidence scores | “Dog at this box; bicycle at that box.” |
| Segmentation | Which pixels belong to each object or region? | Pixel- or region-level masks | “These exact pixels form the dog.” |
Classification is the simplest of the three outputs. Detection adds object-level location. Segmentation adds more detailed boundaries. A system can combine these tasks, but they are not interchangeable. For more background, read Image Classification Explained and Computer Vision Explained.
How object detection works
There is no single universal detector pipeline. Modern systems learn visual features from training images, then use those learned features to predict classes and locations.
Training: learn from annotated examples
A detection dataset commonly pairs each image with a class label and bounding-box coordinates for each annotated object. These reference labels and boxes are called ground truth. The model compares its predictions with them, measures classification and localization errors, and updates its parameters. Annotation rules determine which objects should be marked and how their boxes are drawn.
Preprocessing must keep images and annotations aligned: resizing or flipping an image also requires transforming its boxes. Use validation data to choose model settings and confidence thresholds; reserve a held-out test set for evaluation after those choices. See Training vs Testing Data.
Inference: use the trained detector
At inference, a new image or video frame passes through the trained model to produce candidate classes, boxes, and scores. Ordinary inference does not update the model parameters. Filtering and model-specific post-processing then produce the final detections:
- Process the image. A neural network converts image pixels into useful visual representations.
- Generate object candidates. The detector predicts possible objects through dense image locations, region proposals, learned object queries, or a hybrid approach.
- Predict labels and locations. Each candidate receives class information and a localization prediction such as a bounding box.
- Filter the results. Confidence thresholds and, for many architectures, overlap-based post-processing determine which predictions remain.
Several candidates may describe the same object. Many detectors use Non-Maximum Suppression (NMS): keep a high-scoring candidate and suppress sufficiently overlapping candidates under the chosen rules. This helps reduce duplicates, but aggressive suppression can remove real neighboring objects in a crowded scene. Not every architecture requires NMS; some predict an object set directly.
Architecture families continue to evolve. The durable idea is the task itself: a detector must recognize individual objects and localize them, not merely label the image as a whole.
Detection vs tracking
Detection finds objects in an image or frame. Tracking associates objects across successive frames so a system can follow the same instance over time. A detector may supply observations to a tracker, but boxes in separate frames do not by themselves establish a persistent identity.
Bounding boxes
A bounding box is a rectangle that approximates an object’s location. Systems may store a box as two corner coordinates or as a center point plus width and height. Either representation describes the same basic region.
Localization quality matters. A detector could predict the correct class but draw a box that barely covers the object. Under many evaluation rules, that prediction would not count as a correct match.
Intersection over Union (IoU)
Intersection over Union measures the overlap between a predicted region and a reference region. For bounding boxes, the calculation is:
IoU = area of overlap ÷ area of union
For a simple example, suppose the overlapping portion of two boxes has an area of 40 square units and their combined union has an area of 100 square units. The IoU is 40 ÷ 100, or 0.40. Perfectly aligned boxes have an IoU of 1.00; boxes with no overlap have an IoU of 0.
An evaluation protocol can use an IoU threshold to decide whether a predicted box matches a ground-truth object. Different benchmarks may use different thresholds or average results across several thresholds, so the threshold must always be stated or understood in context.
Precision and recall in object detection
Detection evaluation accounts for both recognition and localization:
- Precision asks what share of reported detections are correct matches.
- Recall asks what share of ground-truth objects were successfully detected.
Changing the confidence threshold can change both metrics. Read Accuracy vs Precision vs Recall for a broader introduction to these measures.
What is mAP?
Mean Average Precision, or mAP, summarizes object-detection performance under a defined evaluation protocol. The progression is: IoU helps decide whether a predicted box matches a reference object; average precision (AP) summarizes the precision–recall relationship for one class as score thresholds vary under those matching rules; mAP averages the class AP values. Some benchmarks also average results over multiple IoU thresholds. A box with sufficient overlap still needs the correct class and a valid match under the protocol.
An mAP value is not meaningful in isolation. Dataset classes, IoU thresholds, object-size ranges, maximum detections, interpolation rules, and implementation details can all affect the reported number. Compare mAP values only when the benchmark and protocol are compatible.
Confidence scores and thresholds
Detectors often assign confidence scores to candidate predictions. A score expresses model support, not certainty, and it is not automatically a calibrated probability that the detection is correct. A confidently wrong prediction is possible. A deployment threshold decides which candidates are shown or acted upon.
- A higher threshold can remove weak predictions and reduce false positives, but it may miss more real objects.
- A lower threshold can retain more real objects and improve recall, but it may also report more false positives.
The right setting depends on the cost of each error. A manufacturing system that must catch rare defects may tolerate more false alarms. A low-risk consumer feature may prioritize fewer distracting false detections. Thresholds should be tuned and validated on data that reflects the intended environment.
Modern object-detection architectures
Modern detection includes several broad architecture families:
- One-stage detectors predict object information directly across image locations and are often designed for efficient inference.
- Two-stage detectors first generate candidate regions and then classify and refine them.
- Transformer-based detectors can formulate detection as set prediction using learned object queries.
- Hybrid systems combine ideas from multiple families.
No family is best for every use. Model choice depends on accuracy requirements, latency, hardware, object size and density, available training data, and the deployment environment.
Real-world applications
- Manufacturing: finding components or visible defects on a production line.
- Retail and inventory: locating products, shelves, or empty spaces.
- Robotics: helping a robot perceive objects it may navigate around or manipulate.
- Traffic analysis: counting and locating vehicles, cyclists, and pedestrians.
- Agriculture: identifying crops, fruit, weeds, or visible signs of stress.
- Medical-imaging research and support: locating candidate regions for expert review.
- Video analytics: detecting objects frame by frame before tracking or event analysis.
Detection is often one component of a larger perception or decision-support system. High-stakes uses require appropriate validation, human oversight, privacy controls, and domain-specific evaluation. A model that performs well on a general benchmark is not automatically safe or effective in a specialized setting.
Common challenges
Small or crowded objects
Small objects contain fewer visual details, and crowded scenes can make neighboring instances difficult to separate. The same category can occupy a few pixels in one image and much of the frame in another. This scale variation makes representative training examples and evaluation across object sizes important.
Occlusion
Objects partly hidden behind other objects may be harder to recognize and localize reliably.
Distribution shift
Changes in cameras, lighting, weather, geography, backgrounds, or object appearance can reduce performance after deployment.
Class imbalance
Rare categories may have poor recall even when aggregate metrics look strong. Per-class results can reveal failures hidden by an average.
Annotation quality
Inconsistent boxes, missing objects, and ambiguous class labels limit what a model can learn and complicate evaluation.
Frequently asked questions
Is object detection the same as image classification?
No. Image classification labels an entire image. Object detection identifies individual objects and predicts a location for each one.
What is a bounding box?
A bounding box is a rectangle that approximates an object’s location in an image. It is commonly stored using corner coordinates or a center point with width and height.
What is IoU?
Intersection over Union measures the overlap between a predicted region and a reference region relative to their combined union area.
What is mAP?
mAP summarizes detection performance across classes under a defined precision–recall and localization protocol. It must be interpreted with its benchmark and evaluation settings.
Is a higher confidence threshold always better?
No. Higher thresholds can reduce false positives but also miss real objects. The useful threshold depends on the application and the consequences of each error type.
Where to learn next
Next: Continue Exploring Computer Vision to connect detection with the broader visual-AI learning path.
For supporting lessons, revisit Image Classification Explained or explore Model Evaluation Metrics Explained.
Technical references
The TorchVision detection tutorial illustrates class-and-box annotations, training, evaluation, and prediction outputs.
- Ren et al., Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.
- Carion et al., End-to-End Object Detection with Transformers.
- Lin et al., Microsoft COCO: Common Objects in Context.