Journal of Scientific and Industrial Research

Open Access Open Access  Restricted Access Subscription Access

Learning How to Detect Salient Objects in Nighttime Scenes


Affiliations
1 School of Computer Science, Sichuan Normal University, Chengdu, China
2 Department of Chongqing University, University of Cincinnati Joint Co-op Institution, Chongqing University, Chongqing, China
3 Department of Health Administration and Policy, George Mason University, Fairfax, Virginia, United States
 

The detection of salient objects in nighttime scene settings is an essential research issue in computer vision. None of the known approaches can accurately anticipate salient objects in the nighttime scenes. Due to the lack of visible light, spatial visual information cannot be accurately perceived by traditional and deep network models. This paper proposed a Mountain Basin Network (MBNet) to identify salient objects for distinguishing the pixel-level saliency of low-light images. To improve the objects localizations and pixel classification performances, the proposed model incorporated a High-Low Feature Aggregation Module (HLFA) to synchronize the information from a high-level branch (named Bal-Net) and a low-level branch (called Mol-Net) to fuse the global and local context, and a Hierarchical Supervision Module (HSM) was embedded to aid in obtaining accurate salient objects, particularly the small ones. In addition, a multi-supervised integration technique was explored to optimize the structure and borders of salient objects. In the meantime, to facilitate more investigation into nighttime scenes and assessment of visual saliency models, we created a new nighttime dataset consisting of thirteen categories and a total of one thousand low-light images. Our experimental results demonstrated that the suggested MBNet model outperforms seven current state-of-the-art methods for salient object detection in nighttime scenes.

Keywords

High-Low Feature Aggregation, Hierarchical Supervision, Multi-Supervised Integration, Nighttime Images, Salient Object Detection.
User
Notifications
Font Size

  • Zhang Z, Cui Z, Xu C, Yan Y, Sebe N & Yang J, Patter-affinitive propagation across depth, surface normal and semantic segmentation, in Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit (IEEE) 2019, 4106–4115.

  • Tang J & Acton S, An image retrieval algorithm using multiple query images, in 7th Int Symp Signal Process Appl (IEEE) 2003, 193–196.

  • Tang J, Sun Q & Agyepong K, An image enhancement algorithm based on a contrast measure in the wavelet domain for screening mammograms, in IEEE Int Conf Imag Process (IEEE) 2007, 29–32.

  • Wang Q, Zhang L & Bertinetto L, Fast online object tracking and segmentation: A unifying approach, in IEEE Conf Comput Vis Pattern Recognit (IEEE) 2018, 1–13.

  • Mu N, Wang H, Zhang Y, Jiang J & Tang J, Progressive global perception and local polishing network for lung infection segmentation of COVID-19 CT images, Pattern Recognit, 120 (2021) 1–12.

  • Liu X, Yuan Q, Wang B, Tang X, Tang J & Shen D, Weakly supervised segmentation of COVID-19 infection with scribble annotation on CT images, Pattern Recognit, 122 (2022) 1–15.

  • He J, Zan Q, and Zhang K, Yu P & Tang J, An evolvable adversarial network with gradient penalty for COVID-19 infection segmentation, Appl Soft Comput, 133 (2021) 1–10.

  • Zhao C, Xu Y, He Z, Tang J, Zhang Y, Han J, Shi Y & Zhou W, Lung segmentation and automatic detection of COVID-19 using radiomic features from chest CT images, Pattern Recognit, 119 (2021) 1–14.

  • He K, Zhang X & Ren S, Deep residual learning for image recognition, in IEEE Conf Comput Vis Pattern Recognit (IEEE) 2019, 30–42.

  • Liu X, Yu A, Wei X, Pan Z & Tang J, Multimodal MR image synthesis using gradient prior and adversarial learning, J Sel Top Signal Process, 14 (2020) 1176–1188.

  • Liu N, Han J & Yang M, PiCANet: Learning pixel-wise contextual attention for saliency detection, in IEEE Conf Comput Vis Pattern Recognit (IEEE) 2018, 3089–3098.

  • Chen S, Tan X, Wang B & Hu X, Reverse attention for salient object detection, in Euro Conf Comput Vis (Computer Vision Foundation) 2018, 234–250.

  • Li X, Song D & Wang B, Hierarchical feature fusion network for salient object detection, IEEE Trans Imag Process, 29 (2020) 9165–9175.

  • Zhang X, Wang T & Qi J, Progressive attention guided recurrent network for salient object detection, in IEEE Conf Comput Pattern Recognit (IEEE) 2018, 18–22.

  • Mu N, Xu X, Zhang X & Lin X, Discrete stationary wavelet transform based saliency information fusion from frequency and spatial domain in low contrast images, Pattern Recognit Lett, 115 (2018) 84–91.

  • Mu N, Xu X, Zhang X & Zhang H, Salient object detection using a covariance-based CNN model for low-contrast images, Neural Comput Appl, 29(8) (2018) 181–192.

  • Xu X, Wang S, Wang Z, Zhang X & Hu R, Exploring image enhancement for salient object detection in low light images. ACM Trans Multimed Comput Commun Appl, 17(1s) (2021) 1–19.

  • Lore K G, Akintayo A & Sarkar S, LLNet: A deep autoencoder approach to natural low-light image enhancement, in IEEE Conf Comput Vis (IEEE) 2017, 650–662.

  • Zhu M, Pan P, Chen W & Yang Y, EEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network, in Proc AAAI Conf Artif Intell, 34(7) (2020), 106–113.

  • Xu K, Yang X, Yin B & Lau R W H, Learning to restore low-light images via decomposition-and-enhancement, in IEEE Conf Compu Vis Pattern Recognit (IEEE) 2020, 2281–2290.

  • Li J, Feng X & Hua Z, Low-light image enhancement via progressive-recursive network, IEEE Trans Circuits Syst Video Technol, 31(11) (2021) 4227–4240.

  • Ronneberger O, Fischer P & Brox T, U-net: Convolutional networks for biomedical image segmentation, in Med Image Comput Comput Assist Interv (Springer, Cham) 2015, 234–241.

  • Badrinarayanan V, Kendal A & Cipolla R, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans Pattern Anal Mach Intell, 39(12) (2017) 2481–2495.

  • Tang J, Millington S, Acton S T, Crandall J & Hurwitz S, Ankle cartilage surface segmentation using directional gradient vector flow snakes, in Int Conf Imag Process (IEEE) 2004, 2745–2748.

  • Lin T, Dollar P, Girshick R, He K, Hariharan B & Belongie S, Feature pyramid networks for object detection, in IEEE Conf Comput Vis and Pattern Recognit (IEEE) 2017, 2117–2125.

  • Jadon S, A survey of loss functions for semantic segmentation, IEEE Conf Comput Intell Bioinformat Computat Biol (CIBCB) (IEEE) 2020, 1–6

  • Lin T, Goyal P, Girshick R, He K & Dollar P, Focal loss for dense object detection, in IEEE Int Conf Comput Vis (IEEE) 2017, 2999–3007.

  • Zhu W, Huang Y, Zeng L, Chen X, Liu Y, Qian Z, Du N, Fan W & Xie X, AnatomyNet: deep learning for fast and fully Automated whole-volume segmentation of head and neck anatomy, arXiv preprint 1808.05238, (2018) 1–13.

  • Lin T, Yuan Z & Sun J, Learning to detect a salient object, IEEE Trans Pattern Anal Mach Intell, 33(2) (2011) 353–367.

  • Yang C, Zhang L & Lu H, Saliency detection via graph-based manifold ranking, in IEEE Conf Compu Vis Pattern Recognit (IEEE) 2013, 3166–3173.

  • Li Y, Hou X & Koch C, The secrets of salient object segmentation, in IEEE Conf Comput Vis Pattern Recognit (IEEE) 2014, 280–287.

  • Li G, Lu H & Wang Y, Visual saliency based on multiscale deep features, in IEEE Conf Comput Vis (2015) 5455–5463.

  • Wang L, Lu H & Wang Y, Learning to detect salient objects with image-level supervision, in IEEE Conf Comput Vis Pattern Recognit (2017) 136–145.

  • Mu N, Xu X & Zhang X, Salient object detection in low contrast images via global convolution and boundary refinement, in IEEE Conf Comput Vis Pattern Recognit Works (2019) 1–9.

  • Wei C, Wang W & Yang W, Deep retinex decomposition for low-light enhancement, arXiv preprint 1808.04560 (2019) 1–12.

  • Loh Y & Chan C, Deep residual learning for image recognition, in IEEE Conf Comput Vis Pattern Anal Mach Intell (2016) 770–778.

  • Qin X, Zhang Z, Huang C, Gao C, Dehghan M & Jagersand M, Basnet: Boundary-aware salient object detection, in IEEE Conf Compu Vis Patt Recog (2019) 7479–7489.

  • Jun W, Wang S & Huang Q, F³Net: fusion, feedback, and focus for salient object detection, in Proc AAAI Conf Artifi Intell, 34(7) (2020) 12321–12328.

  • Sun H, Jun C & Liu N, MPI: Multi-receptive and parallel integration for salient object detection, in IEEE Conf on Comput Vis Pattern Recognit (2021) 1–10.

  • Sun H, Bain Y & Liu N, Multi-scale edge-based U-shape network for salient object detection, in IEEE Conf Comput Vis Pattern Recognit (2021) 1–14.

  • Valanarasu J, Sindagi V, Hacihaliloglu I & Patel V, Kiu-net: Towards accurate segmentation of biomedical images using over-complete representations, in Med Image Comput Comput Assist Interv (Springer, Cham) 2020, 363–373.


Abstract Views: 115

PDF Views: 77




  • Learning How to Detect Salient Objects in Nighttime Scenes

Abstract Views: 115  |  PDF Views: 77

Authors

Nan Mu
School of Computer Science, Sichuan Normal University, Chengdu, China
Jinjia Guo
Department of Chongqing University, University of Cincinnati Joint Co-op Institution, Chongqing University, Chongqing, China
Jinshan Tang
Department of Health Administration and Policy, George Mason University, Fairfax, Virginia, United States

Abstract


The detection of salient objects in nighttime scene settings is an essential research issue in computer vision. None of the known approaches can accurately anticipate salient objects in the nighttime scenes. Due to the lack of visible light, spatial visual information cannot be accurately perceived by traditional and deep network models. This paper proposed a Mountain Basin Network (MBNet) to identify salient objects for distinguishing the pixel-level saliency of low-light images. To improve the objects localizations and pixel classification performances, the proposed model incorporated a High-Low Feature Aggregation Module (HLFA) to synchronize the information from a high-level branch (named Bal-Net) and a low-level branch (called Mol-Net) to fuse the global and local context, and a Hierarchical Supervision Module (HSM) was embedded to aid in obtaining accurate salient objects, particularly the small ones. In addition, a multi-supervised integration technique was explored to optimize the structure and borders of salient objects. In the meantime, to facilitate more investigation into nighttime scenes and assessment of visual saliency models, we created a new nighttime dataset consisting of thirteen categories and a total of one thousand low-light images. Our experimental results demonstrated that the suggested MBNet model outperforms seven current state-of-the-art methods for salient object detection in nighttime scenes.

Keywords


High-Low Feature Aggregation, Hierarchical Supervision, Multi-Supervised Integration, Nighttime Images, Salient Object Detection.

References