Publications

Xia, Y; He, W; Huang, Q; Yin, GY; Liu, WB; Zhang, HY (2024). CRformer: Multi-modal data fusion to reconstruct cloud-free optical imagery. INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION, 128, 103793.

Abstract
Cloud contamination is a common problem in Earth observation that hinders various remote sensing applications. To address this problem, recent studies have employed deep neural networks and multi -modal data fusion to reconstruct cloud -free optical imagery. However, this task faces many challenges, such as: (1) the scarcity of suitable multi -modal datasets; (2) the ineffective use of feature correlations; and (3) the limited applicability of existing models. To overcome these challenges, this study proposes a novel solution that fuses high -spatial SAR and low -spatial optical data to reconstruct high -quality cloud -free multi -spectral optical products. First, a curated benchmark dataset, named SMILE-CR, is created with a realistic cloud simulation strategy. The SMILE-CR serves as a global and multi -modal cloud removal dataset for the Landsat8 sensor, with Sentinel -1 and MODIS data as additional supplementary data. Second, a Transformer -based cloud removal network, abbreviated as CRformer, is developed with two novel modules: multi -head dense and sparse attention and multi -scale gated-dconv feed -forward network. The CRformer achieves global attention while suppressing the weak correlations and enhancing the multi -scale cloud features by filtering out invalid features. The performance of the proposed method is evaluated through extensive experiments. The results show that the CRformer surpasses the state-of-the-art cloud removal methods with significant improvements in both quantitative and qualitative metrics. The fusion of MODIS and Sentinel -1 data is shown to be effective and necessary in reconstructing Landsat-8 observations. Moreover, the CRformer model can be readily applied to reconstruct time -series cloud -free Landsat-8 products in Wuhan city, which can improve the average accuracy of land cover mapping by over 3%.

DOI:
10.1016/j.jag.2024.103793

ISSN:
1872-826X