Downloads

Viedma, I. A., Alonso-Caneiro, D., Read, S. A., Charng, J., Chen, F. K., Mackey, D. A., & Collins, M. J. Impact of Image Complexity on the Segmentation of Ophthalmic Images Using Deep Learning Methods. Journal of Bio-Optics. 2025. doi: Retrieved from https://w3.sciltp.com/journals/jbo/article/view/648
Volume 1, Issue 1, 2025
Copyright (c) 2025 by the authors.
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Article

Impact of Image Complexity on the Segmentation of Ophthalmic Images Using Deep Learning Methods

Ignacio A. Viedma 1, David Alonso-Caneiro 1,2,*, Scott A. Read 1, Jason Charng 3,4,  Fred K. Chen 4,5,6,7, David A. Mackey 3,5 and Michael J. Collins 1

1 Contact Lens and Visual Optics Laboratory, Optometry and Vision Science, QUT, Brisbane, 4059 QLD, Australia

2 School of Science, Technology and Engineering, University of Sunshine Coast, Sunshine Coast, 4502 QLD, Australia

3 Centre for Ophthalmology and Visual Science, The University of Western Australia, Perth, 6009 WA, Australia

4 Department of Optometry, School of Allied Health, University of Western Australia, Perth, 6009 WA, Australia

5 Lions Eye Institute, Nedlands, 6009 WA, Australia

6 Ophthalmology, Department of Surgery, University of Melbourne, East Melbourne, 3002 VIC, Australia

7 Royal Victorian Eye and Ear Hospital, East Melbourne, 3002 VIC, Australia

* Correspondence: dalonsocaneiro@usc.edu.au

Received: 10 December 2024; Revised: 10 February 2025; Accepted: 13 March 2025; Published: 1 April 2025

Abstract: Advances in medical imaging segmentation using deep learning (DL) have facilitated the development of a wide range of models based on different architectures. For example, the U-Net has become one of the most widely used architectures in the field. Due to its popularity, various modifications to the original U-Net architecture have been proposed, with the aim to improve the segmentation performance. Most studies utilizing ophthalmic images, such as optical coherence tomography (OCT) and scanning laser ophthalmoscopy (SLO), have employed U-Net methods and their variations for segmentation. Given the multitude of U-Net variations, selecting the optimum model for ophthalmic image segmentation may be challenging, and other factors that may impact the model’s segmentation performance, such as image complexity and its impact on model selection, have been largely unexplored. Thus, in this study, the segmentation performance of three models, including a baseline U-Net, a popular U-Net variation (U-Net++), and a segmentation architecture with a different approach (DeepLabV3), are compared, analyzing how these different methods may vary for OCT and SLO datasets with various levels of segmentation task complexity. To analyze the effect of image complexity on segmentation performance, several metrics are extracted, including the image entropy of the datasets at the pixel level, the texture level, and the global features level. The results demonstrate a relationship between the complexity of the images in a dataset and the performance of the segmentation model used for the specific task. Data complexity may serve as a metric to inform DL model selection or aid in the early design process. 

Keywords:

optical coherence tomography scanning laser ophthalmoscopy semantic segmentation deep learning image complexity

References

  1. Lateef, F.; Ruichek, Y. Survey on semantic segmentation using deep learning techniques. Neurocomputing 2019, 338, 321–348.
  2. Ulku, I.; Akagunduz, E. A survey on deep learning-based architectures for semantic segmentation on 2D images. Appl. Artif. Intell. 2021, 36, e2032924.
  3. Kugelman, J.; Allman, J.; Read, S.A.; et al. A comparison of deep learning U-Net architectures for posterior segment oct retinal layer segmentation. Sci. Rep. 2022, 12, 14888.
  4. Venhuizen, F.G.; van Ginneken, B.; Liefers, B.; et al. Robust total retina thickness segmentation in optical coherence tomography images using convolutional neural networks. Biomed. Opt. Express 2017, 8, 3292–3316.
  5. Hassan, T.; Usman, A.; Akram, M.U.; et al. Deep learning based automated extraction of intra-retinal layers for analyzing retinal abnormalities. In Proceedings of the 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom), Ostrava, Czech Republic, 17–20 September 2018.
  6. Kiaee, F.; Fahimi, H.; Rabbani, H. Intra-retinal layer segmentation of optical coherence tomography using 3D fully convolutional networks. In Proceedings of the International Conference on Image Processing, ICIP, Athens, Greece, 7–10 October 2018; pp. 2795–2799.
  7. Liu, Y.; Ren, G.; Yang, G.; et al. Fully convolutional network and graph-based method for co-segmentation of retinal layer on macular OCT images. In Proceedings of the 24th International Conference on Pattern Recognition (ICPR), Beijing, China, 20–24 August 2018; pp. 3081–3085.
  8. Pekala, M.; Joshi, N.; Liu, T.A.; et al. Deep learning based retinal OCT segmentation. Comput. Biol. Med. 2019, 114, 103445.
  9. Ruan, Y.; Xue, J.; Li, T.; et al. Multi-phase level set algorithm based on fully convolutional networks (FCN-MLS) for retinal layer segmentation in SD-OCT images with central serous chorioretinopathy (CSC). Biomed. Opt. Express 2019, 10, 3987–4002.
  10. Apostolopoulos, S.; De Zanet, S.; Ciller, C.; et al. Pathological OCT retinal layer segmentation using branch residual U-shape networks. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Quebec City, QC, Canada, 11–13 September 2017; pp. 294–301.
  11. He, Y.; Carass, A.; Yun, Y.; et al. Towards topological correct segmentation of macular OCT from cascaded FCNs. Lect. Notes Comput. Sci. 2017, 10554, 202–209.
  12. Roy, A.; Conjeti, S.; Karri, S.; et al. Relaynet: Retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed. Opt. Express 2017, 8, 3627–3642.
  13. Hamwood, J.; Alonso-Caneiro, D.; Read, S.; et al. Effect of patch size and network architecture on a convolutional neural network approach for automatic segmentation of OCT retinal layers. Biomed. Opt. Express 2018, 9, 3049–3066.
  14. Kugelman, J.; Alonso-Caneiro, D.; Read, S.A.; et al. Automatic choroidal segmentation in OCT images using supervised deep learning methods. Sci. Rep. 2019, 9, 13298.
  15. Gao, K.; Niu, S.; Ji, Z.; et al. Double-branched and area-constraint fully convolutional networks for automated serous retinal detachment segmentation in SD-OCT images. Comput. Methods Programs Biomed. 2019, 176, 69–80.
  16. Li, M.-X.; Yu, S.-Q.; Zhang, W.; et al. Segmentation of retinal fluid based on deep learning: Application of threedimensional fully convolutional neural networks in optical coherence tomography images. Int. J. Ophthalmol. 2019, 12, 1012.
  17. Lee, C.S.; Tyring, A.J.; Deruyter, N.P.; et al. Deeplearning based, automated segmentation of macular edema in optical coherence tomography. Biomed. Opt. Express 2017, 8, 3440–3448.
  18. Tennakoon, R.; Gostar, A.K.; Hoseinnezhad, R.; et al. Retinal fluid segmentation in OCT images using adversarial loss based convolutional neural networks. In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA, 4–7 April 2018; pp. 1436–1440.
  19. Kang, S.H.; Park, H.S.; Jang, J.; et al. Deep neural networks for the detection and segmentation of the retinal fluid in OCT images. In Proceedings of the Retinal OCT Fluid Challenge (RETOUCH), Quebec City, Canada, 10–14 September 2017; pp. 9–14.
  20. Cunefare, D.; Fang, L.; Cooper, R.F.; et al. Open source software for automatic detection of cone photoreceptors in adaptive optics ophthalmoscopy using convolutional neural networks. Sci. Rep. 2017, 7, 6620.
  21. Chen, Y.; He, Y.; Wang, J.; et al. Automated cone photoreceptor cell identification in confocal adaptive optics scanning laser ophthalmoscope images based on object detection. J. Innov. Opt. Health Sci. 2022, 15, 2250001.
  22. Chen, Y.; He, Y.; Wang, J.; et al. Deeplab and bias field correction based automatic cone photoreceptor cell identification with adaptive optics scanning laser ophthalmoscope images. Wirel. Commun. Mob. Comput. 2021, 2021, 2034125.
  23. Hamwood, J.; Alonso-Caneiro, D.; Sampson, D.M.; et al. Automatic detection of cone photoreceptors with fully convolutional networks. Transl. Vis. Sci. Technol. 2019, 8, 10.
  24. Zhang, Q.; Sampani, K.; Xu, M.; et al. Aoslo-net: A deep learning-based method for automatic segmentation of retinal microaneurysms from adaptive optics scanning laser ophthalmoscopy images. Transl. Vis. Sci. Technol. 2022, 11, 7.
  25. Sułot, D.; Alonso-Caneiro, D.; Ksieniewicz, P.; et al. Glaucoma classification based on scanning laser ophthalmoscopic images using a deep learning ensemble method. PLoS ONE 2021, 16, e0252339.
  26. Viedma, I.A.; Alonso-Caneiro, D.; Read, S.A.; et al. Deep learning in retinal optical coherence tomography (OCT): A comprehensive survey. Neurocomputing 2022, 507, 247–264.
  27. Siddique, N.; Paheding, S.; Elkin, C.P.; et al. U-Net and its variants for medical image segmentation: A review of theory and applications. IEEE Access 2021, 9, 82031–82057.
  28. Du, G.; Cao, X.; Liang, J.; et al. Medical image segmentation based on U-Net: A review. J. Imaging Sci. Technol. 2020, 64, 1–12.
  29. Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S.S.; et al. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, 17–21 October 2016; pp. 424–432.
  30. Oktay, O.; Schlemper, J.; Folgoc, L.L.; et al. Attention U-Net: Learning where to look for the pancreas. arXiv 2018, arXiv:1804.03999.
  31. Alom, M.Z.; Hasan, M.; Yakopcic, C.; et al. Recurrent residual convolutional neural network based on U-Net (r2U-Net) for medical image segmentation. arXiv 2018, arXiv:1802.06955.
  32. Zhou, Z.; Siddiquee, M.M.R.; Tajbakhsh, N.; et al. Unet++: A nested U-Net architecture for medical image segmentation. In Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Granada, Spain, 20 September 2018; pp. 3–11.
  33. Rahane, A.A.; Subramanian, A. Measures of complexity for large scale image datasets. In Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan, 19–21 February 2020; pp. 282–287.
  34. Khan, T.M.; Naqvi, S.S.; Meijering, E. Leveraging image complexity in macro-level neural network design for medical image segmentation. Sci. Rep. 2022, 12, 22286.
  35. Chen, L.-C.; Papandreou, G.; Schroff, F.; et al. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587.
  36. Read, S.A.; Collins, M.J.; Vincent, S.J.; et al. Choroidal thickness in myopic and nonmyopic children assessed with enhanced depth imaging optical coherence tomography. Investig. Ophthalmol. Vis. Sci. 2013, 54, 7578–7586.
  37. Kugelman, J.; Alonso-Caneiro, D.; Chen, Y.; et al. Retinal boundary segmentation in stargardt disease optical coherence tomography images using automated deep learning. Transl. Vis. Sci. Technol. 2020, 9, 12.
  38. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; pp. 234–241.
  39. Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440.
  40. He, K.; Zhang, X.; Ren, S.; et al. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778.
  41. Szegedy, C.; Ioffe, S.; Vanhoucke, V.; et al. Inception-v4, inception-resnet and the impact of residual connections on learning. Proc. AAAI Conf. Artif. Intell. 2017, 31, 4278–4284.
  42. Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258.
  43. Zhou, Z.; Siddiquee, M.M.R.; Tajbakhsh, N.; et al. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 2019, 39, 1856–1867.
  44. Huang, G.; Liu, Z.; Van Der Maaten, L.; et al. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708.
  45. Chen, L.-C.; Papandreou, G.; Kokkinos, I.; et al. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv 2014, arXiv:1412.7062.
  46. Holschneider, M.; Kronland-Martinet, R.; Morlet, J.; et al. A real-time algorithm for signal analysis with the help of the wavelet transform. In Proceedings of the Wavelets: Time-Frequency Methods and Phase Space Proceedings of the International Conference, Marseille, France, 14–18 December 1987; Springer: Berlin/Heidelberg, Germany, 1990; pp. 286–297.
  47. Sermanet, P.; Eigen, D.; Zhang, X.; et al. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv 2013, arXiv:1312.6229.
  48. Papandreou, G.; Kokkinos, I.; Savalle, P.-A. Modeling local and global deformations in deep learning: Epitomic convolution, multiple instance learning, and sliding window detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 390–399.
  49. Giusti, A.; Cires, D.C.; Masci, J.; et al. Fast image scanning with deep max-pooling convolutional neural networks. In Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, VIC, Australia, 15–18 September 2013; pp. 4034–4038.
  50. Chen, L.-C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 40, 834–848.
  51. He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 1904–1916.
  52. Iakubovskii, P. Segmentation Models Pytorch. 2019. Available online: https://github.com/qubvel/segmentation_models.pytorch (accessed on 27th of March 2025).