Application of Image Fusion Method Based on Structural Tensor in Marine Exploration

MA Xiaoyi; CHEN Yihong; WANG Fei; XIE Shuo

doi:10.11993/j.issn.2096-3920.2024-0066

Article Contents

Article Navigation > Journal of Unmanned Undersea Systems > 2025 > Accepted Manuscript

MA Xiaoyi, CHEN Yihong, WANG Fei, XIE Shuo. Application of Image Fusion Method Based on Structural Tensor in Marine Exploration[J]. Journal of Unmanned Undersea Systems. doi: 10.11993/j.issn.2096-3920.2024-0066

Citation:

MA Xiaoyi, CHEN Yihong, WANG Fei, XIE Shuo. Application of Image Fusion Method Based on Structural Tensor in Marine Exploration[J]. Journal of Unmanned Undersea Systems. doi: 10.11993/j.issn.2096-3920.2024-0066

Citation:

PDF( 1298 KB)

Application of Image Fusion Method Based on Structural Tensor in Marine Exploration

doi: 10.11993/j.issn.2096-3920.2024-0066

MA Xiaoyi^{1, 2},
CHEN Yihong^{1, 2, 3},
WANG Fei^{1, 2},
XIE Shuo^{1, 2}

1.
China Ship Scientific Research Center, Wuxi, 214026, China
2.
Taihu Laboratory of Deepsea Technological Science, Wuxi, 214026, China
3.
School of Aeronautics and Astronautics, Zhejiang University, Hangzhou, 310058, China

Received Date: 2024-04-09
Accepted Date: 2024-07-10
Rev Recd Date: 2024-06-20

Available Online: 2025-01-22

Abstract

Abstract

Single sensor cannot achieve well results in marine detection. Infrared and visible light have strong complementarity, and fusing them can obtain high-quality fused images, enabling more accurate and comprehensive perception of maritime targets. However, existing fusion methods have not been applied in the field of maritime detection, and they lack specificity, poor fusion effects, and a lack of deep learning datasets for maritime fusion. This article studies the deep learning image fusion method based on structural tensors, improves and optimizes the characteristics of maritime targets, adds multi-scale convolution, and fuses images according to channels, aiming to obtain high-quality color fusion images with significant targets and comprehensive information. The data set used for marine fusion is constructed from the data collected in the Taihu Lake Lake. The collected data are used for experiments, and a variety of evaluation indicators are comprehensively selected for comparative simulation experiments. The research results indicate that the improved image fusion method performs better than the original algorithm in six indicators, and its overall performance is better than the other ten commonly used image fusion algorithms. The generalization of the improved method has been verified on other public datasets. The improved image fusion method based on structural tensors has excellent performance in sea perception, with fusion results highlighting target features and better fusion performance than other methods.
- marine exploration,
- image fusion,
- deep learning,
- structure tensor

FullText(HTML)

References(34)

References

[1]	邱志明, 孟祥尧, 马焱, 等. 海上无人系统发展及关键技术研究[J]. 中国工程科学, 2023, 25(3): 74-83. QIU Z M, MENG X Y, MA Y. et al. Development and Key Technologies of Maritime Unmanned Systems[J]. Strategic Study of Chinese Academy of Engineerng, 2023, 25(3): 74-83.
[2]	贲可荣, 王斌. 海洋装备智能化与智能化装备思考[J]. 江苏科技大学学报(自然科学版), 2021, 35(2): 1-11. BEN K R, WANG B. Thinking on the intellectualization of marine equipment and marine intelligent equipment[J]. Journal of Jiangsu University of Science and Technology (Natural Science Edition), 2021, 35(2): 1-11.
[3]	张蕾. 红外与可见光图像融合技术研究[D]. 中国科学院研究生院(长春光学精密机械与物理研究所), 2015.
[4]	TOET A. Image fusion by a ratio of low-pass pyramid[J]. Pattern Recognition Letters, 1989: 245-253.
[5]	PAJARES G, Jesús Manuel de la Cruz. A wavelet-based image fusion tutorial[J]. Pattern Recognition, 2004, 37(9): 1855-1872.
[6]	杨晓慧, 金海燕, 焦李成. 基于DT-CWT的红外与可见光图像自适应融合[J]. 红外与毫米波学报, 2007(6): 419-424. doi: 10.3321/j.issn:1001-9014.2007.06.005 YANG X H, JIN H Y, JIAO L C. Adaptive image fusion algorithm for infrared and visible light images based on DT-CWT[J]. Journal of Infrared and Millimeter Waves, 2007(6): 419-424. doi: 10.3321/j.issn:1001-9014.2007.06.005
[7]	LI S , KANG X , HU J. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875.
[8]	BAVIRISETTI D P, DHULI R. Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform[J]. IEEE Sensors Journal, 2015, 16(1): 203-209.
[9]	KUMAR B K S. Image fusion based on pixel significance using cross bilateral filter[J]. Signal, Image and Video Processing, 2015, 5: 1193-1204.
[10]	MA J, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109. doi: 10.1016/j.inffus.2016.02.001
[11]	LI H, WU X. Infrared and visible image fusion using latent low-rank representation[J]. Cornell University- arXiv, Cornell University - arXiv, 2018.
[12]	BAVIRISETTI D P, XIAO G, ZHAO J, et al. Multi-scale guided image and video fusion: A fast and efficient approach[J]. Circuits, Systems, and Signal Processing, 2019, 38(1): 5576-5605.
[13]	BAVIRISETTI D P, DHULI R. Two-scale image fusion of visible and infrared images using saliency detection[J]. Infrared Physics & Technology, 2016, 76: 52-64.
[14]	罗迪. 无人机平台下的可见光与热红外图像融合和目标检测研究[D]. 南京航空航天大学, 2021.
[15]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Neural Information Processing Systems, Neural Information Processing Systems, 2017.
[16]	MA C J J. FusionGAN: A generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019: 11-26
[17]	XU H, LIANG P, YU W, et al. Learning a generative model for fusing infrared and visible images via conditional generative adversarial network with dual discriminators. [C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao, China: IJCAI, 2019.
[18]	PRABHAKAR K R, SRIKAR V S, BABU R V. DeepFuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//2017 IEEE International Conference on Computer Vision. Venice: ICCV, 2017.
[19]	LI H, WU X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018: 2614-2623.
[20]	LI H, WU X J, KITTLER J. RFN-Nest: An end-to-end residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 2: 72-86.
[21]	TANG W, HE F, LIU Y. TCCFusion: An infrared and visible image fusion method based on transformer and cross correlation[J]. Pattern Recognition, 2023: 109295.
[22]	LI H, WU X J, KITTLER J. Infrared and visible image fusion using a deep learning framework[C]//2018 24th International Conference on Pattern Recognition. Beijing, CHINA: ICPR, 2018.
[23]	LONG Y, JIA H, ZHONG Y, et al. RXDNFuse: A aggregated residual dense network for infrared and visible image fusion[J]. Information Fusion, 2021, 69(1): 128-141.
[24]	MA J, TANG L, XU M, et al. STDFusionNet: An infrared and visible image fusion network based on salient target detection[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70(1): 1-13.
[25]	LIU J, FAN X, JIANG J, et al. Learning a Deep Multi-scale Feature Ensemble and an Edge-attention Guidance for Image Fusion[J/OL]. IEEE Transactions on Circuits and Systems for Video Technology, 2022: 105-119.
[26]	TANG L, ZHANG H, XU H, et al. Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity[J]. Information Fusion, 2023: 101870.
[27]	JUNG H, KIM Y, JANG H, et al. Unsupervised deep image fusion with structure tensor representations[J]. IEEE Transactions on Image Processing, 2020, 29(99): 3845-3858.
[28]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA: CVPR, 2016.
[29]	IOFFE S, SZEGEDY C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[J]. arXiv: Learning, arXiv: Learning, 2015.
[30]	MISRA D. Mish: A self regularized non-monotonic neural activation function[J]. 2019.
[31]	LIN M, CHEN Q, YAN S. Network In Network[J]. In Proceedings of the IEEE International Conference on Learning Representations, 2014.
[32]	WANG Z, BOVIK A C. A universal image quality index[J]. IEEE Signal Processing Letters, 2002, 9(3): 81-84. doi: 10.1109/97.995823
[33]	唐霖峰, 张浩, 徐涵, 等. 基于深度学习的图像融合方法综述[J]. 中国图象图形学报, 2023, 28(1): 3-36. doi: 10.11834/jig.220422
[34]	TOET A. The TNO multiband image data collection[J]. Data in Brief, 2017, 15: 249-251. doi: 10.1016/j.dib.2017.09.038