首页 >  , Vol. , Issue () : -

摘要

全文摘要次数: 114 全文下载次数: 58
引用本文:

DOI:

10.11834/jrs.20243249

收稿日期:

2023-06-29

修改日期:

2024-03-06

PDF Free   EndNote   BibTeX
从光学到SAR:基于多级跨模态对齐的SAR图像舰船检测算法
何佳月1, 宿南1, 徐从安2, 尹璐3, 廖艳苹1, 闫奕名1
1.哈尔滨工程大学 信息与通信工程学院;2.海军航空大学 信息融合研究所;3.北京市遥感信息研究所
摘要:

合成孔径雷达(SAR)舰船检测是近年来的研究热点。然而,与光学图像不同,SAR成像的特点会导致不直观的特征表示。此外,由于SAR图像数据量不足,现有的基于大量标记SAR图像的方法可能难以达到较好的检测效果。为了解决这些问题,本文提出了一种基于多级跨模态对齐的SAR图像舰船检测算法MCMA-Net(Multi-level Cross-Modality Alignment Network),通过将光学模态中丰富的知识迁移到SAR模态来增强SAR图像的特征表示。该算法首先设计了一个基于邻域-全局注意力的特征交互网络NGAN(Neighborhood-Global Attention Network),通过对骨干网络的浅层特征采用邻域注意力机制进行局部交互、对深层特征采取全局自注意力机制进行全局上下文交互,在兼顾全局上下文建模能力的同时,提升局部特征的编码能力,使得网络在不同层级更合理的关注相应的信息,从而能够促进后续的多级别模态对齐。其次,本文设计了一个多级模态对齐模块MLMA(Multi-level Modality Alignment),通过从局部级别到全局级别再到实例级别的对两种模态不同隐含空间中的特征进行对齐,促进模型有效地学习模态不变特征,缓解了光学图像和SAR图像之间的模态鸿沟,实现了从光学模态到SAR模态的知识传输。大量的实验证明我们的算法优于现阶段的检测算法,取得了最好的实验结果。

From optical to SAR: A SAR ship detection algorithm based on multi-level cross-modality alignment
Abstract:

Synthetic Aperture Radar (SAR) ship detection is a hot topic in recent years. However, unlike optical images, the characteristics of SAR imaging result in non-intuitive feature representations. Additionally, due to the limited amount of SAR image data, existing methods that rely on a large number of annotated SAR images may hard to achieve satisfactory results. To address these issues, a SAR ship detection algorithm called MCMA-Net, which is based on multi-level cross-modality alignment is proposed in this paper, which enhances the feature representation of SAR images by transferring rich knowledge from optical modality. First, we propose a neighborhood-global attention-based feature interaction network (NGAN), which employs a neighborhood attention mechanism to enable local interaction of low-level features and a global self-attention mechanism to capture global context from high-level features. While taking into account the ability of global context modeling, the encoding ability of local features is improved, NGAN makes the network pay more attention to the corresponding information at different levels and can promote the subsequent multi-level modality alignment. Secondly, we propose a multi-level modality alignment module (MLMA), which aligns the features in different hidden spaces of the two modalities from the local level to the global level and then to the instance level. It promotes the model to effectively learn the modality invariant features and alleviates the modality gap between the optical and SAR image, which realizes the knowledge transmission from the optical modality to the SAR modality. Plenty of experiments show that our algorithm is superior to the current detection algorithm and achieves the best experimental results.

本文暂时没有被引用!

欢迎关注学报微信

遥感学报交流群