Objectives Image segmentation is one of the important subjects in the field of image processing and computer vision. Image segmentation has a wide range of applications in robot perception, autonomous driving, medical image analysis, scene understanding, video surveillance, virtual reality and augmented reality, but it also faces many challenges. For example, it is necessary to rely on the context to obtain more precise results in complex scenes. Deep learning-based image segmentation algorithms have made great progress, but they also face many problems, especially most algorithms lack the ability of combining image context, and they ignore the role and influence of the contextual information on the segmentation contour, which limits the improvement of algorithm performance. For this reason, we propose a contour-based image segmentation approach, which uses a progressive dynamic graph network to deform the contour.
Methods Specifically, according to the topology of object contour, we first sample the vertices from the contour to convert them into a dynamic graph. We abandon the disorder of the graph and the uncertainty of the adjacency relationship, to construct a graph with a fixed topology. And then we perform inference learning by diffusing contextual information of the object points. We also progressively accumulates historical learning experience to update the contour graph dynamically. We adopt an end-to-end method to update the graph and predict object position, and encapsulate them into a closed-loop learning process. The proposed approach fully takes into account context and the historical learning experience. It also makes up for the shortcomings of the pixel-wise segmentation methods and traditional active contour models.
Results Experiments on Cityscapes, KINS, SBD datasets show that the effectiveness and speed of the proposed approach in real-time instance segmentation. Specifically, the dynamic graph model achieves the best performances on Cityscapes and SBD datasets, which are 34.4% and 55.3% average precision（AP） respectively, and 30.5% AP on KINS dataset. It also achieves a better fitting effect on KINS dataset in terms of visualization results. Compared with deep snake model, the segmentation contours extracted by dynamic graph model fit the real boundary better and the contours are smoother. In addition, in terms of running speed, the dynamic graph model also obtains best results on three datasets, which are 3.8, 7.6 and 13.1 frame/s respectively. The proposed method has an average improvement of 0.5 frame/s on three datasets when compared with deep snake model.
Conclusions Progressive dynamic graph makes better use of the topology of object contour and the characteristics of dynamic update. By disseminating context information and historical information, this method solves the problem that segmentation algorithms lack the ability of combining context. Future research can also face the following aspects: Designing a more efficient and robust network architecture, considering the objects around the target to combine the context information of scene in the image,and attempting to resolve the issue of poor convergence on non-convex objects or concave regions.
DownLoad: