DL之Panoptic Segmentation：Panoptic Segmentation(全景分割)的简介(论文介绍)、全景分割挑战简介、案例应用等配图集合之详细攻略
DL之Panoptic Segmentation：Panoptic Segmentation(全景分割)的简介(论文介绍)、全景分割挑战简介、案例应用等配图集合之详细攻略
相关文章
DL之Panoptic Segmentation：Panoptic Segmentation(全景分割)的简介(论文介绍)、全景分割挑战简介、案例应用等配图集合之详细攻略
DL之Panoptic Segmentation：Panoptic Segmentation(全景分割)的全景分割挑战的简介
Panoptic Segmentation(全景分割)的简介(论文介绍)
本论文源自FaceBook的研究人员。
Abstract
We propose and study a task we name panoptic segmentation (PS). Panoptic segmentation unifies the typically distinct tasks of semantic segmentation (assign a class label to each pixel) and instance segmentation (detect and segment each object instance). The proposed task requires generating a coherent scene segmentation that is rich and complete, an important step toward real-world vision systems. While early work in computer vision addressed related image/scene parsing tasks, these are not currently popular, possibly due to lack of appropriate metrics or associated recognition challenges. To address this, we propose a novel panoptic quality (PQ) metric that captures performance for all classes (stuff and things) in an interpretable and unified manner. Using the proposed metric, we perform a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task. The aim of our work is to revive the interest of the community in a more unified view of image segmentation.dual stuff-and-thing nature of PS. A number of instance segmentation approaches including [28, 2, 3, 18] are designed to produce non-overlapping instance predictions and could serve as the foundation of such a system. (2) Since a PS cannot have overlapping segments, some form of higherlevel 'reasoning’ may be beneficial, for example, based on extending learnable NMS [7, 16] to PS. We hope that the panoptic segmentation task will invigorate research in these areas leading to exciting new breakthroughs in vision. Finally we note that the panoptic segmentation task was featured as a challenge track by both the COCO [25] and Mapillary Vistas [35] recognition challenges and that the proposed task has already begun to gain traction in the community (e.g. [23, 48, 49, 27, 22, 21, 17] address PS).
我们提出并研究了一项称为全景分割(PS)的任务。泛光分割统一了语义分割(为每个像素分配一个类标签)和实例分割(检测和分割每个对象实例)这两个典型的不同任务。提出的任务需要生成一个连贯的场景分割是丰富和完整的，一个重要的步骤，向现实世界的视觉系统。虽然早期的计算机视觉工作解决了相关的图像/场景解析任务，但这些任务目前并不流行，这可能是因为缺乏适当的度量标准或相关的识别挑战。为了解决这个问题，我们提出了一种新的全景质量(PQ)矩阵，它以一种可解释和统一的方式捕获所有类(东西和东西)的性能。使用提议的度量，我们对现有的三个数据集上的PS的人和机器性能进行了严格的研究，揭示了关于该任务的有趣见解。我们的工作目标是唤起社会各界对图像分割的兴趣，以更统一的视角进行图像分割。许多实例分割方法，包括[28,2,3,18]，旨在产生非重叠的实例预测，并可作为这样一个系统的基础。(2)由于一个PS不能有重叠的片段，某种形式的高层次的“推理”可能是有益的，例如，基于将可学习的NMS[7,16]扩展到PS，我们希望全景分割任务能够活跃这些领域的研究，从而在视觉方面带来令人兴奋的新突破。最后，我们注意到，COCO[25]和map腋下远景[35]识别挑战都将全光分割任务作为一个挑战轨迹，并且所提出的任务已经开始在社区中获得关注(例如，[23, 48, 49, 27, 22, 21, 17]地址PS)。
Future of Panoptic Segmentation
Our goal is to drive research in novel directions by inviting the community to explore the new panoptic segmentation task. We believe that the proposed task can lead to expected and unexpected innovations. We conclude by discussing some of these possibilities and our future plans.
我们的目标是通过邀请社区来探索新的全景分割任务，从而将研究推向新的方向。我们认为，拟议的任务可以导致预期的和意想不到的创新。最后，我们讨论了其中一些可能性和我们未来的计划。
Motivated by simplicity, the PS 'algorithm’ in this paper is based on the heuristic combination of outputs from topperforming instance and semantic segmentation systems. This approach is a basic first step, but we expect more interesting algorithms to be introduced. Specifically, we hope to see PS drive innovation in at least two areas: (1) Deeply integrated end-to-end models that simultaneously address the dual stuff-and-thing nature of PS. A number of instance segmentation approaches including [28, 2, 3, 18] are designed to produce non-overlapping instance predictions and could serve as the foundation of such a system. (2) Since a PS cannot have overlapping segments, some form of higherlevel 'reasoning’ may be beneficial, for example, based on extending learnable NMS [7, 16] to PS. We hope that the panoptic segmentation task will invigorate research in these areas leading to exciting new breakthroughs in vision.
本论文的PS“算法”以简单为动机，基于top performance实例输出和语义分割系统的启发式组合。这种方法是基本的第一步，但我们希望引入更多有趣的算法。具体地说,我们希望看到PS驱动创新至少在两个方面:(1)深入集成的端到端模型,同时解决双重stuff-and-thing PS的性质。许多实例分割方法包括(28日,2、3、18)是用来产生重叠实例预测,可以作为这样的一个系统的基础。(2)由于一个 PS不能有重叠的片段，某种形式的高层次的“推理”可能是有益的，例如，基于将可学习的NMS[7,16]扩展到PS，我们希望泛光分割任务能够活跃这些领域的研究，从而在视觉方面带来令人兴奋的新突破。
Finally we note that the panoptic segmentation task was featured as a challenge track by both the COCO [25] and Mapillary Vistas [35] recognition challenges and that the proposed task has already begun to gain traction in the community (e.g. [23, 48, 49, 27, 22, 21, 17] address PS).
最后，我们注意到，COCO[25]和Mapillary Vistas [35]识别挑战都将全景分割任务作为一个挑战轨迹，并且所提出的任务已经开始在社区中获得关注(例如，[23, 48, 49, 27, 22, 21, 17]地址PS)。
论文
Alexander Kirillov, KaimingHe, Ross Girshick, Carsten Rother, Piotr Dollár.
Panoptic Segmentation
https://arxiv.org/pdf/1801.00868.pdf
0、论文简介
CV之IS：计算机视觉之图像分割(Image Segmentation)算法的挑战任务、算法演化、目标检测和图像分割的对比
Panoptic Segmentation全景分割挑战简介
DL之Panoptic Segmentation：Panoptic Segmentation(全景分割)的全景分割挑战的简介
Panoptic Segmentation(全景分割)的案例应用
更新……