Apparel Image Segmentation with Mask R-CNN and GrabCut

A talk by Ankith Gunapal. Computer Vision is changing the face of fashion with numerous applications. One such application is extracting apparel attributes such as color, texture, pattern by analyzing apparel images. To obtain these features, we need to first recognize apparel in an image. Image segmentation is a technique used to achieve this. Instance Segmentation is a subtype of image segmentation that identifies each instance of an object in a given image. Mask R-CNN is a popular deep learning based instance segmentation algorithm for images. It produces a pixel wise segmentation mask of objects in an image along with a bounding box. It has proven to be extremely effective in a wide variety of projects, especially when the objects are homogeneous in shape. In certain scenarios, the image background can creep into the object’s mask produced by Mask R-CNN. Apparel, especially women’s apparel, can be considered heterogeneous in shape. Obtaining the segmentation mask of apparel using Mask R-CNN produces mixed results. GrabCut is a computer vision algorithm to extract foreground from an image. One of the inputs to the algorithm is specifying an approximate region of the object of interest. Once we specify a mask or bounding box, GrabCut runs iteratively and can produce pretty accurate foreground segmentation mask. The presentation will cover both these algorithms and explain how we can combine Mask R-CNN and GrabCut to obtain good quality segmentation mask for apparel.