Describe Features of Computer Vision Workloads on Azure
The four analysis tasks
- Image classification - one or more labels describing the whole image ("dog").
- Object detection - a class label plus a bounding box for each instance, and more than one object can be returned.
- Semantic segmentation - a pixel-level mask per class, so the exact outline is known; instance segmentation separates individual objects of the same class.
- OCR - reading printed or handwritten text, with positions, from images and PDFs.
Azure services to know
Azure AI Vision provides prebuilt capabilities: image tagging, captioning, OCR, spatial analysis and face detection. Custom Vision lets you train your own image classification or object detection models from labelled images. Azure AI Face provides detection (where a face is) and, under a restricted-access review, verification and identification - but face detection alone does not disclose identity, emotion or age with certainty. Azure AI Document Intelligence extracts fields from invoices, receipts, IDs and custom forms. Azure AI Video Indexer analyses video and audio.
Traps that appear in questions
"How many objects and where" always means object detection, not classification. Custom Vision offers classification and object detection only - not segmentation. Captioning gives a sentence; tagging gives keywords. Predicting a single attribute of a person from an image is a limited-access or restricted scenario, so the compliant answer is usually detection, not inference.
Worked example
A warehouse wants a camera to flag each empty pallet slot on a shelf and draw a rectangle around it. One image contains many slots, so the workload is object detection trained with Custom Vision; classification would return a single label for the whole photo and could not count or locate the slots.