Creating a high quality dataset for computer vision is essential to having strong model performance. In addition to collecting images that are as similar to your deployed conditions as possible, labeling images carefully and accurately is essential.
In order to label images effectively, the below post walks through tactics to ensure your dataset is as high quality as possible. While the below best practices are generally true, it is important to note that labeling instructions are highly dependent on the nature of the task at hand. Moreover, images labeled for one task, may not be suitable for another task – re-labeling is not uncommon. It is best to think of a dataset and its labels as something alive: constantly changing and improving to fit the task at hand.
With that in mind, lets walkthrough a few tips.
1. Label Every Object of Interest in Every Image
Computer vision models are built to learn what patterns of pixels correspond to an object of interest. Because of this, if we’re training a model to identify an object, we need to label every appearance of that object in our images. If we do not label the object in some images, we will be introducing false negatives to our model.
2. Label the Entirety of an Object
Our bounding boxes should enclose the entirety of an object of interest. Labeling only a portion of the object confuses our model for what a full object constitutes.
3. Label Occluded Objects
Occlusion is when an object is partially out of view in an image due to something blocking it in a photo. It is best to label even occluded objects. Moreover, it is commonly best practice to label the occluded object as if it were fully visible – rather than drawing a bounding box for only the partially visible portion of the object.
4. Create Tight Bounding Boxes
Bounding boxes should be tight around the objects of interest. (But, you should never have a box so tight that it is cutting off a portion of the object.) Tight bounding boxes are critical to helping our model learn, precisely, which pixels constitute an object of interest vs irrelevant portions of an image.
5. Create Specific Label Names
When determining a given object’s label name, it is better to error on the side of being more specific rather than less. It is always easier to remap label classes to be more general, whereas being more specific requires relabeling. For example, imagine you are building a dog detector. While every object of interest is a dog, it may be wise to create a class for labrador and poodle. In initial model building, our labels could be combined to be dog. But, if we had started with dog and later realized having individual breeds is important, we would have to relabel our dataset altogether.
6. Maintain Clear Labeling Instructions
Inevitably, we will need to add more data to our dataset – this is a key element to model improvement. Tactics like active learning ensure that we spend our time labeling intelligently. As such, having clear, shareable, and repeatable labeling instructions is essential for both our future selves and coworkers to create and maintain high quality datasets.
Many of the techniques we’ve discussed here should be included: label the entirety of an object, make labels tight, label all objects, etc. It is always better to err on the side of more specificity than less.
7. Use These Labeling Tools
Now that we know how to label effectively, how should we label our data? If we’re labeling ourselves, we can use tools like CVAT, LabelImg, RectLabel (for Mac), etc to add and maintain labels.
When we’re ready to scale up our labeling efforts either with teammates or an outsourced workforce, we can pick from a marketplace of labeling services like AWS, Scale, and others. (Reach out to us if you’d like to setup a test to see which service is best for your data.)
As always, happy building!