【泡泡一分钟】挑战性光照条件下的视觉里程计多模态跟踪框架

每天一分钟，带你读遍机器人顶级会议文章

标题：Multimodal tracking framework for visual odometry in challenging illumination conditions

作者：Axel Beauvisage, Kenan Ahiska, Nabil Aouf

来源：2020 IEEE International Conference on Robotics and Automation (ICRA)

编译：余旭东

审核：柴毅，王靖淇

这是泡泡一分钟推送的第 631 篇文章，欢迎个人转发朋友圈；其他机构或自媒体如需转载，后台留言申请授权

摘要

视觉里程计和定位的研究大多是在可见光条件下求解的，其中光照是一个关键因素。电磁波谱的其他部分正被研究，以在极端光照条件下求解。特别地，多谱段的设置是令人感兴趣的，因为它们能同时提供不同谱段的信息。但是，这种相机设置的主要挑战在于生成的图像之间缺少相似性，导致传统的立体匹配技术显得过时。

这项工作研究一种应用于视觉里程计的同时处理不同波谱的图像的新方法。尤其关注的是可见光和长波红外（LWIR）波谱，它们的像素强度之间的不同之处是最多的。我们提出了一种新的多模态单目视觉里程计（MMS-VO），同时提取特征，但是只有提供跟踪质量最好的相机被用于估计运动。视觉里程计通过加窗的光束调整框架实现，当场景本质发生变化时选择不同的相机。而且，根据视差选择适当的关键帧使得运动估计过程是抗差的。

算法在一系列可见光-红外数据集上测试，数据集来自真实场景中驾驶的汽车。结果表明，特征提取能够采用同一组参数在不同模态中实现。此外，多模态单目视觉里程计能提供较好的视觉里程计轨迹，因为当某个相机不能工作时另一个可以补偿。

图1.单一模态的单目视觉里程计算法流程图

表1 每次迭代进行野值剔除之后可见光、红外光以及被选择模态的剩余点的数量

表2 每个序列MMS-VO选择的图像数量

图2 MMS-VO在序列1和序列2中生成的轨迹

表3 MMS-VO和真值（GNSS）之间的误差比较

图3 序列3中的轨迹，a是可见光VO和红外VO单独的轨迹估计，b是进行模态选择之后的轨迹估计

表4 序列3单独模态和多模态之间的误差比较

图4 p-LK失效的例子以及对应的p-LK跟踪结果，每个蓝线表示当前帧和上一帧中特征点的位置

表5 处理多模态图像对花费的时间

Abstract

Research on visual odometry and localisation is largely dominated by solutions developed in the visible spectrum, where illumination is a critical factor. Other parts of the electromagnetic spectrum are currently being investigated to generate solutions dealing with extreme illumination conditions. Multispectral setups are particularly interesting as they provide information from different parts of the spectrum at once. However, the main challenge of such camera setups is the lack of similarity between the images produced, which makes conventional stereo matching techniques obsolete.

This work investigates a new way of concurrently processing images from different spectra for application to visual odometry. It particularly focuses on the visible and Long Wave InfraRed (LWIR) spectral bands where dissimilarity between pixel intensities is maximal. A new Multimodal Monocular Visual Odometry solution (MMS-VO) is presented. With this novel approach, features are tracked simultaneously, but only the camera providing the best tracking quality is used to estimate motion. Visual odometry is performed within a windowed bundle adjustment framework, by alternating between the cameras as the nature of the scene changes. Furthermore, the motion estimation process is robustifified by selecting adequate keyframes based on parallax.

The algorithm was tested on a series of visible-thermal datasets, acquired from a car with real driving conditions. It is shown that feature tracking could be performed in both modalities with the same set of parameters. Additionally, the MMS-VO provides a superior visual odometry trajectory as one camera can compensate when the other is not working.