Remote Sensing and GIS Applications in Environmental Sciences

Implementation and comparison of spectral and spatial quality of pixel-based satellite image fusion methods

Document Type : Original Article

Authors

1 Department of Geomatics, School of Marand Engineering, University of Tabriz,

2 Department of Geomatics, Marand Faculty of Engineering, University of Tabriz, Tabriz, Iran,

Abstract

In this research, 15 conventional image fusion techniques were applied and compared in four different groups. In addition, 10 distinct spectral-spatial criteria were investigated in four different modes to assess the outcomes. In the first case, every concrete criterion was considered, and the results revealed that, with the exception of the RVS and Gramshmit methods, which preserve both spectral and spatial information, the rest of the mentioned methods, while preserving spatial information, fail to preserve spectral information. In the second scenario, the average of seven homogeneous spectral criteria in two categories—minimum value and maximum value—was considered. In the maximum spectral criterion, LMVM, Gramshmit, SVR, and Ehler methods achieved the best performance, and in the minimum spectral criterion, Ehler, SVR, SFIM, and IHS methods ranked highest. In the third condition, the spatial criteria revealed that the LMM, ISVR, Brovey, and PCA algorithms perform best in terms of spatial information preservation. They were assessed jointly in the fourth scenario, taking the average of the lowest and highest spectral-spatial requirements into account. The findings revealed that the maximum mode, according to the LMM, LMVM, RVS, Ehler, Gramshmit, and SVR techniques, and the minimum mode, according to the Ehler, SVR, SFIM, and IHS methods, had the best performance in maintaining the spectral-spatial information. Finally, the results revealed that the SVR, Ehler, and Gramshmit algorithms perform the best in terms of maintaining spectral-spatial information when compared to other methods, preserving around 80 to 95% of the information.

Keywords

Main Subjects

در تصاویر سنجش از دور بدلیل محدودیت سیگنال به نویز سنجنده، امکان اخذ تصاویر با دقت طیفی و مکانی بالا مقدور نیست .هدف این تحقیق پیاده سازی و بررسی جامع طیفی - مکانی روشهای مختلف ادغام تصویر در سطح پیکسل است. بر این اساس ،11 روش مرسوم ادغام تصاویر در چهار گروه شامل روشهای مبتنی بر انتقال، ترکیب محاسباتی، مبتنی بر فیلتر در حوزه زمان- مکان و فرکانس و روشهای آماری ادغام تصاویر پیاده سازی و مقایسه گردید. همچنین جهت ارزیابی کیفیت نتایج ده معیار مختلف کیفیت طیفی و مکانی در چهار حالت مختلف مورد تجزیه- تحلیل قرار گرفتند. در حالت اول، تک تک پارامترهای در نظر گرفته شدند که نتایج این حالت نشان داد، بجز روش RVS  و Gramshmit که هم اطلاعات طیفی و هم اطلاعات مکانی را در تصاویر ادغام شده حفظ میکند، بقیه روشهای مذکور با اینکه اطلاعات مکانی را در تصاویر ادغام شده حفظ میکند در مقابل، در حفظ اطلاعات طیفی عملکرد ضعیفی دارند. در حالت دوم، میانگین تمامی هفت معیار طیفی هم جهت در دو دسته حداقل و حداکثر مقدار در نظر گرفته شدند  که به ترتیب در معیار طیفی حداکثر، روشهای SVR، Gramshmit ،LMVM  و Ehler  و معیار طیفی حداقل روشهای SFIM ،SVR ،Ehler  و IHS  بهترین عملکرد را را با حفظ 01 درصد اطلاعات طیفی داشتند. در حالت سوم، معیار مکانی هم جهت هم نشان داد، روشهای Brovey، ISVR ، LMM و PCA به ترتیب بهترین عملکرد را در حفظ اطلاعات مکانی دارند. در حالت چهارم بررسی، با در نظر گرفتن میانگین معیارهای طیفی و مکانی هم جهت حداقل و حداکثر ،بصورت توام مورد بررسی قرار گرفتند. نتایج این حالت نشان داد، میانگین کیفیت طیفی- مکانی در حالت حداکثر، به ترتیب روشهای Gramshmit ،Ehler ،RVS ،LMVM  ،LMM و SVR در حالت حداقل، به ترتیب روشهای  SFIM ،SVR ،Ehler و IHS در حفظ اطلاعات طیفی و مکانی در تصاویر ادغام شده بهترین عملکرد را داشتهاند. مجوع چهار حالت بررسی نشان داد به ترتیب روشهای Ehler ،SVR و Gramshmit در حفظ اطلاعات طیفی- مکانی در تصاویر ادغام شده بهترین عملکرد را نسبت به سایر روشها دارند و تقریباً از 01 تا 01 درصد اطلاعات را حفظ میکنند .

1-Abbas, H.K., Al-Saleh, A.H. and Al-Zuky, A.A., (2019). Optical Images Fusion Based on Linear Interpolation Methods. Iraqi Journal of Science, 60(4): 924-936.
2-Aiazzi, B., Alparone, L., Baronti, S. and Selva, M., (2012). Twenty-five years of pansharpening. Signal and Image Processing for Remote Sensing: 533-548.
3-AL-HELALI, A.H.M., ZRAQOU, J.S., ALKHADOUR, W. and AL-NUEMI, A., (2021). A NEW HYBRID MULTI-FOCUS IMAGE-FUSION USING DMWT WITH FFT
TRANSFORMS. Journal of Theoretical and Applied Information Technology, 99)01(1.
4-Al-Wassai, F.A., Kalyankar, N. and Al-Zaky, A.A., (2011). The statistical methods of pixelbased image fusion techniques. ArXiv preprint arXiv: 1108.3250.
5-Belgiu, M. and Stein, A., (2019). Spatiotemporal image fusion in remote sensing. Remote sensing, 11 :818(7).
6-Bhavana, V. and Krishnappa, H., (2015). Multi-modality medical image fusion using discrete wavelet transform. Procedia Computer Science, 70: 625-631.
7-Chai, P., Luo, X. and Zhang, Z., (2017). Image fusion using quaternion wavelet transform and multiple features. IEEE access, 5: 6724-6734.
8-Cornet, Y., de Béthune, S., Binard, M., Muller, F., Legros, G. and Nadasdi, I., (2001). RS data fusion by local mean and variance matching algorithms: their respective efficiency in a complex urban context, IEEE/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas (Cat. No. 01EX482). IEEE, pp. 105-111.
9-DadrasJavan, F., Samadzadegan, F. and Fathollahi, F., (2018). Spectral and spatial quality assessment of IHS and wavelet based pan-sharpening techniques for high resolution satellite imagery. Advances in image and video processing, 6(2): 1.
10-Diao, W., Zhang, F., Wang, H., Wan, W., Sun, J. and Zhang, K., (2022). HLF-Net: Pansharpening Based on High and Low Frequency Fusion Networks. IEEE Geoscience and Remote Sensing Letters.
11-Ehlers, M., (2004). Spectral characteristics preserving image fusion based on Fourier domain filtering, Remote sensing for environmental monitoring, GIS applications, and geology IV. SPIE, pp. 1-13.
12-Fei, R., Zhang, X., Li, W., Xiong, J .and Du, F., (2021). A new pansharpening method based on the sparse representation of classified injected details over a featured dictionary. Remote Sensing Letters, 12(8): 808-818.
13-Gangkofner, U.G., Pradhan, P.S. and Holcomb, D.W., (2007). Optimizing the high-pass filter addition technique for image fusion. Photogrammetric Engineering & Remote Sensing, 73(9): 1107-1118.
 
                                                        کاربرد سنجش از دور و GIS در علوم محیطی، شماره 1، سال دوم، زمستان 1011، صص 10-42
Application of remote sensing and GIS in environmental sciences, Vol 2, No. 5, Winter 2023, pp. 27-54
14-Ghadjati, M., Moussaoui, A. and Boukharouba, A., (2019). A novel iterative PCA–based pansharpening method. Remote sensing letters, 10(3): 264-2.37
15-Ghahremani, M. and Ghassemian, H., (2016). Nonlinear IHS: A promising method for pansharpening. IEEE Geoscience and Remote Sensing Letters, 13(11): 1606-1610.
16-Gogineni, R., Chaturvedi, A. and BS, D.S., (2021). A variational pan-sharpening algorithm to enhance the spectral and spatial details. International Journal of Image and Data Fusion, 12(3): 242-264.
17-Han, S., Li, H. and Gu., H., (2008). The study on image fusion for high spatial resolution remote sensing images. Int. Arch. Photogramm. Remote Sens. Spat .Inf. Sci. XXXVII. Part B, 7: 1159-1164.
18-Hasanlou, M. and Saradjian, M.R., (2016). Quality assessment of pan-sharpening methods in high-resolution satellite images using radiometric and geometric index. Arabian Journal of Geosciences, 9(1): 1-10.
19-Huang, W ,.Xiao, L., Wei, Z., Liu, H. and Tang, S., (2015). A new pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5): 1037-1041.
20-Jagalingam, P. and Hegde, A.V., (2015). A review of quality metrics for fused image. Aquatic Procedia, 4: 133-142.
21-Javan, F.D., Samadzadegan, F., Mehravar, S., Toosi, A., Khatami, R. and Stein, A., (2021). A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery. ISPRS journal of photogrammetry and remote sensing, 171: 101-117.
22-Klonus, S. and Ehlers, M., (2007). Image fusion using the Ehlers spectral characteristics preservation algorithm. GIScience & Remote Sensing, 44(2): 93-116.
23-Laben, E., (2000). Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening (Gram-Schmidt). US Patent: USA, Eastman Kodak Company.
24-Li, S., Kang, X., Fang, L., Hu, J. and Yin, H., (2017). Pixel-level image fusion: A survey of the state of the art. Information Fusion, 33: 100-112.
25-Li, Y. and Jiang, S., 2020 .Multi-focus image fusion using geometric algebra based discrete fourier transform. Ieee Access, 8: 60019-60028.
26-Liu, X., Liu, Q. and Wang, Y., (2020a). Remote sensing image fusion based on two-stream fusion network. Information Fusion, 55: 1-15.
27-Liu, Y., Wang, L., Cheng, J., Li, C. and Chen, X., (2020b). Multi-focus image fusion: A survey of the state of the art. Information Fusion, 64: 71-91.
28-May, R.I., (2018). Verification of sea ice drift data obtained from remote sensing information, IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium. IEEE, pp. 7344-7347.
29-Mhangara, P., Mapurisa, W. and Mudau, N., (2020). Comparison of image fusion techniques using satellite pour l’Observation de la Terre (SPOT) 6 satellite imagery. Applied Sciences, (5):1881.
30-Palsson, F., Sveinsson, J.R. and Ulfarsson, M.O., (2013). A new pansharpening algorithm based on total variation. IEEE Geoscience and Remote Sensing Letters, 11(1): 318-322.
31-Pushparaj, J. and Hegde, A.V., (2017). Evaluation of pan-sharpening methods for spatial and spectral quality. Applied Geomatics, 9(1): 1-12.
32-Rodríguez-Esparragón, D., Marcello, J., Eugenio, F., García-Pedrero, A. and Gonzalo-Martín, C., (2017). Object-based quality evaluation procedure for fused remote sensing imagery. Neurocomputing, 255: 40-51.
33-Rogge, D.M., Rivard, B., Zhang, J., Sanchez, A., Harris, J. and Feng, J., (2007). Integration of spatial–spectral information for the improved extraction of endmembers. Remote Sensing of Environment, 110(3): 287-303.
34-Shahdoosti, H.R., (2017) .MS and PAN image fusion by combining Brovey and wavelet methods. ArXiv preprint arXiv:1701.01996.
35-Singh, M., Singh, R. and Ross, A., (2019). A comprehensive overview of biometric fusion. Information Fusion, 52: 187-205.
36-Sulaiman, A.G., Elashmawi, W.H. and Eltaweel, G.S., (2021). IHS-based pan-sharpening technique for visual quality improvement using KPCA and enhanced SML in the NSCT domain. International Journal of Remote Sensing, 42(2): 537-566.
37-Toutin, T. and Cheng, P., (2002). QuickBird–a milestone for high resolution mapping. Earth Observation Magazine, 11(4): 14-18.
38-Tu, T.-M., Lee, Y.-C., Huang, P.S. and Chang, C.-P., (2006). Modified smoothing-filter-based technique for IKONOS-QuickBird image fusion. Optical Engineering, 45(6): 066201.
39-Wald, L., (2000). Quality of high resolution synthesised images: Is there a simple criterion?, Third conference" Fusion of Earth data: merging point measurements, raster maps and remotely sensed images". SEE/URISCA, pp. 99-103.
40-Wald, L., (2002). Data fusion: definitions and architectures: fusion of images of different spatial resolutions. Presses des MINES.
41-Wang, L., Cao, X. and Chen, J., (2008). ISVR: an improved synthetic variable ratio method for image fusion. Geocarto International, 23(2): 155-165.
42-Wang, Z. and Bovik, A.C. (2002). A universal image quality index. IEEE signal processing letters, 9(3): 81-84.
43-WG, I., (2008). Quality assessment for multi-sensor multi-date image fusion.
 
                                                        کاربرد سنجش از دور و GIS در علوم محیطی، شماره 1، سال دوم، زمستان 1011، صص 10-42
Application of remote sensing and GIS in environmental sciences, Vol 2, No. 5, Winter 2023, pp. 27-54
44-Xiao, G., Bavirisetti, D.P., Liu, G., Zhang, X., Xiao, G., Bavirisetti, D.P., Liu, G. and Zhang , X., (2020). Decision-level image fusion. Image Fusion: 149-170.
45-Xu, L., Du, J., Hu, Q. and Li, Q., (2013). Feature-based image fusion with a uniform discrete curvelet transform. International Journal of Advanced Robotic Systems, 10(5): 255.
46-Yakhdani, M.F. and Azizi, A., (2010). Quality assessment of image fusion techniques for multisensor high resolution satellite images (case study: IRS-P5 and IRS-P6 satellite images). na.
47-Yang, Y., Huang, S., Gao, J. and Qian, Z., (2014). Multi-focus image fusion using an effective discrete wavelet transform based algorithm. Measurement science review, 14(2): 102.
48-Yilmaz, V., Serifoglu Yilmaz, C., Güngör, O. and Shan, J., (2020). A genetic algorithm solution to the gram-schmidt image fusion. International Journal of Remote Sensing, 41(4): 14581485.
49-Yilmaz, V., Yilmaz, C.S. and Gungor, O., (2021). Genetic algorithm-based synthetic variable ratio image fusion. Geocarto International, 36(9): 989-1006.
50-Yuan, Q., Wei, Y., Meng, X., Shen, H. and Zhang, L., (2018). A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 978-989.
51-Zhou, J., Civco, D.L. and Silander, J., (1998). A wavelet transform method to merge Landsat TM and SPOT panchromatic data. International journal of remote sensing, 19(4): 743-757.
Remote Sensing and GIS Applications in Environmental Sciences
Volume 2, Issue 5 - Serial Number 5
March 2023
Pages 54-27
  • Receive Date: 01 January 2023
  • Revise Date: 30 January 2023
  • Accept Date: 15 March 2023