Study of ensemble methods applied to semantic segmentation masks

Semantic segmentation is a fundamental technique for automating the detection of objects of interest in images. However, its application faces a major challenge: the need for a large amount of annotated data for training, a process that requires significant effort and time, making it unfeasible in some cases, such as applications in the healthcare field, especially those aimed at the detection of cancerous nuclei. In this context, the application of ensemble techniques can be a viable solution to improve the performance of semantic segmentation models while requiring a smaller amount of data. In view of this, this work proposes and evaluates an ensemble approach based on stacking to combine the masks generated by the U-Net and DeepLabV3+ models, combined with the VGG16, ResNet50, and MobileNetV3Large backbones. The proposal uses one or more convolutional layers to combine the masks in a way that is sensitive to the local context of the pixels, allowing the model to learn to exploit the strengths and weaknesses of each network dynamically. The objective is to generate a final mask capable of outperforming the results obtained by each individual model and also by the traditional ensemble methods evaluated. The results demonstrate that the proposed approach is statistically indistinguishable from the best static ensembles evaluated. However, it is possible that the limited number of examples available for training the stacking model limited its generalization capability.