Review
BibTex RIS Cite

Semantic Based Image Retrieval-A Survey

Year 2021, Volume: 5 Issue: 2, 445 - 457, 30.12.2021

Akif Gaşi Tolga Ensari Mustafa Dağtekin

Abstract

As a result of rapid technological development, operating with massive data has become a common situation. There is a need for machine learning to process these data and extract meaningful information, and make a decision from them. Current studies related to identifying objects from the image are driven to Semantic- Based Image Retrieval. The studies done in this field aim to dismiss the discrepancies among the low-level color, shape, texture characteristics and picture recognition by people that are extracted from images by machines known as the Semantic Gap, that are signified as high-level concepts. Therefore, definite ontologies are created to determine characteristics of the concept of a particular domain and show the relationship between them by advancing the research on this area. Through ontologies, information is transformed into a structure so computers can process and create a meaningful relationship between information. In this study, a compilation on Semantic-Based Image Retrieval – SBIR is done. SBIR aims to overcome the bottleneck faced in the search operations created by Content-Based Image Retrieval (CBIR) and shown as a Semantic Gap. In the studies done, significant progress in problem-solving through the use of the Ontology concept is observed.

Keywords

Image Retrieval Semantic Gap Deep Learning Semantic Based Image Retrieval

References

  • Alkhawlani, M., Elmogy, M., & El Bakry, H. (2015). Text-based, content-based, and semantic-based image retrievals: A survey. In International Journal of Computer and Information Technology (ISSN: 2279–0764) (Vol. 4, Issue 01).
  • Alpaydın, E. (2004). Introduction to machine learning. MIT Press.
  • Alzu’bi, A., Amira, A., & Ramzan, N. (2015). Semantic content-based image retrieval: A comprehensive study. Journal of Visual Communication and Image Representation, 32, 20–54. https://doi.org/10.1016/j.jvcir.2015.07.012
  • Alzu’bi, A., Amira, A., & Ramzan, N. (2017). Content-based image retrieval with compact deep convolutional features. In Neurocomputing (Vol. 249, pp. 95–105). https://doi.org/10.1016/j.neucom.2017.03.072
  • Ashraf, R., Ahmed, M., Jabbar, S., Khalid, S., Ahmad, A., Din, S., & Jeon, G. (2018). Content Based Image Retrieval by Using Color Descriptor and Discrete Wavelet Transform. Journal of Medical Systems, 42(3). https://doi.org/10.1007/s10916-017-0880-7
  • Aslandogan, Y. A., & Yu, C. T. (1999). Techniques and systems for image and video retrieval. In IEEE Transactions on Knowledge and Data Engineering (Vol. 11, Issue 1, pp. 56–63). https://doi.org/10.1109/69.755615
  • Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
  • Bouchakwa, M., Ayadi, Y., & Amous, I. (2020). Multi-level diversification approach of semantic-based image retrieval results. Progress in Artificial Intelligence, 9(1), 1–30. https://doi.org/10.1007/s13748-019-00195-x
  • Chen, H., Guo, A. Bin, Ni, W., & Cheng, Y. (2020). Improving the representation of image descriptions for semantic image retrieval with RDF. Journal of Visual Communication and Image Representation, 73(August 2019), 102934. https://doi.org/10.1016/j.jvcir.2020.102934
  • De Geus, D., Meletis, P., & Dubbelman, G. (2020). Fast panoptic segmentation network. IEEE Robotics and Automation Letters, 5(2), 1742–1749. https://doi.org/10.1109/LRA.2020.2969919
  • Deserno, T. M., Antani, S., & Long, R. (2009). Ontology of gaps in content-based image retrieval. In Journal of Digital Imaging (Vol. 22, Issue 2, pp. 202– 215). https://doi.org/10.1007/s10278-007-9092-x
  • Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016). Deep learning for visual understanding: A review. In Neurocomputing (Vol. 187, pp. 27–48). https://doi.org/10.1016/j.neucom.2015.09.116
  • Li, Y., Wang, Y., & Huang, X. (2007). A relation-based search engine in Semantic Web. IEEE Transactions on Knowledge and Data Engineering, 19(2), 273–281. https://doi.org/10.1109/TKDE.2007.18
  • Liu, Y., Zhang, D., Lu, G., & Ma, W. Y. (2007). A survey of content-based image retrieval with high-level semantics. Pattern Recognition, 40(1), 262–282. https://doi.org/10.1016/j.patcog.2006.04.045
  • Long, J., Shelhamer, E., & Darrell, T. (2015). Fully Convolutional Networks for Semantic Segmentation. In IEEE. https://doi.org/10.1109/CVPR.2015.7298965
  • Ma, H., Zhu, J., Lyu, M. R. T., & King, I. (2010). Bridging the semantic gap between image contents and tags. IEEE Transactions on Multimedia, 12(5), 462–473. https://doi.org/10.1109/TMM.2010.2051360
  • Mezaris, V., Kompatsiaris, I., & Strintzis, M. G. (2003). An ontology approach to object-based image retrieval. Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429), 3, II-511–514. https://doi.org/10.1109/ICIP.2003.1246729
  • Minu, R. I., & Thyagharajan, K. K. (2014). Semantic rule based image visual feature ontology creation. International Journal of Automation and Computing, 11(5), 489–499. https://doi.org/10.1007/s11633-014-0832-3
  • Ngo, T. G., Ngo, Q. T., & Nguyen, D. D. (2016). Image Retrieval with relevance feedback using SVM active learning. International Journal of Electrical and Computer Engineering, 6(6), 3238–3246. https://doi.org/10.11591/ijece.v6i6.11631
  • Noh, H., Hong, S., & Han, B. (2015). Learning Deconvolution Network for Semantic Segmentation (Vol. 1). https://doi.org/10.1109/ICCV.2015.178
  • Pang, Y., Li, Y., Shen, J., & Shao, L. (2019). Towards bridging semantic gap to improve semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, 2019-Octob(Iccv), 4229–4238. https://doi.org/10.1109/ICCV.2019.00433
  • Parsons, S. (2009). A Semantic Web Primer, Second Edition by Antoniou Grigoris and Harmelen Frank van, MIT Press, 288 pp.. In The Knowledge Engineering Review (Vol. 24, Issue 4). https://doi.org/10.1017/s0269888909990117
  • Rizwan I Haque, I., & Neubert, J. (2020). Deep learning approaches to biomedical image segmentation. In Informatics in Medicine Unlocked (Vol. 18). https://doi.org/10.1016/j.imu.2020.100297
  • Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000). Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1349–1380. https://doi.org/10.1109/34.895972
  • Song, K., Li, F., Long, F., Wang, J., & Ling, Q. (2018). Discriminative Deep Feature Learning for Semantic-Based Image Retrieval. IEEE Access, 6, 44268– 44280. https://doi.org/10.1109/ACCESS.2018.2862464
  • Tzelepi, M., & Tefas, A. (2018). Deep convolutional learning for Content Based Image Retrieval. In Neurocomputing (Vol. 275, pp. 2467–2478). https://doi.org/10.1016/j.neucom.2017.11.022
  • Wang, Q., Lai, J., Claesen, L., Yang, Z., Lei, L., & Liu, W. (2020). A novel feature representation: Aggregating convolution kernels for image retrieval. Neural Networks, 130, 1–10. https://doi.org/10.1016/j.neunet.2020.06.010
  • Wu, Q. (2020). Image retrieval method based on deep learning semantic feature extraction and regularization softmax. Multimedia Tools and Applications, 79(13–14), 9419–9433. https://doi.org/10.1007/s11042-019-7605-5
  • Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2020). Deep multimodal fusion for semantic image segmentation: A survey. Image and Vision Computing, 104042. https://doi.org/https://doi.org/10.1016/j.imavis.2020.104042
  • Zhao, R., & Grosky, W. I. (2002). Narrowing the semantic gap - Improved text-based web document retrieval using visual features. IEEE Transactions on Multimedia, 4(2), 189–200. https://doi.org/10.1109/TMM.2002.1017733
  • Zhu, H. (2020). Massive-scale image retrieval based on deep visual feature representation. Journal of Visual Communication and Image Representation, 70. https://doi.org/10.1016/j.jvcir.2019.102738
  • WordNet. Retrieved from https://wordnet.princeton.edu, (22.11.2020)

Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme

Year 2021, Volume: 5 Issue: 2, 445 - 457, 30.12.2021

Akif Gaşi Tolga Ensari Mustafa Dağtekin

Abstract

Bilgisayar teknolojisinin hızlı gelişmesi sonucunda çok büyük miktarlarda verilerle çalışmak olağan bir durum haline gelmiştir. Bu verilerin işlenmesi, verilerden anlamlı bilgiler çıkarılması ve kararlar alınması için makine öğrenmesine ihtiyaç duyulmaktadır. Görüntü içerisinden nesnelerin algılanmasına yönelik son zamanlarda yapılan çalışmalar özellikle Anlamsal Tabanlı Görüntü Erişimi alanına doğru yönelmektedir. Bu alanda yapılan çalışmalar ile Anlamsal Boşluk olarak adlandırılan ve görüntülerden makineler tarafından çıkarılan düşük düzeydeki renk, şekil, doku (color, shape, texture) özellikleri ile insanlar tarafından resimlerden algılanan ve yüksek düzey olarak ifade edilen kavramlar arasındaki uyuşmazlıkların giderilmesine yoğunlaşmaktadır. Bu amaçla, belirli bir bilgi alanına (domain) ait kavramların özelliklerini ve aralarındaki ilişkileri göstermek için iyi tanımlanmış ontolojiler oluşturulmakta ve arama işlemi bu yönde ilerlemektedir. Ontolojiler kullanılarak bilgiler bilgisayarların işleyebileceği biçime dönüştürülmekte ve bilgiler arasında anlamlı ilişkiler oluşturulabilmektedir. Bu çalışmada Anlamsal Tabanlı Görüntü Erişimi (Semantic Based Image Retrieval - SBIR) üzerine bir derleme yapılmıştır. SBIR ile amaç İçerik Tabanlı Görüntü Erişimi (Content Based Image Retrieval - CBIR) ile yapılan arama işlemlerinde karşılaşılan ve Anlamsal Boşluk (Semantic Gap) olarak ifade edilen darboğazın aşılmasıdır. Yapılan çalışmalarda Ontoloji (Ontology) kavramının kullanılmasıyla problemin çözümünde önemli bir gelişme yaşandığı gözlemlenmiştir.

Keywords

Görüntü Alma Anlamsal Boşluk Derin Öğrenme Semantik Görüntü Alma

References

  • Alkhawlani, M., Elmogy, M., & El Bakry, H. (2015). Text-based, content-based, and semantic-based image retrievals: A survey. In International Journal of Computer and Information Technology (ISSN: 2279–0764) (Vol. 4, Issue 01).
  • Alpaydın, E. (2004). Introduction to machine learning. MIT Press.
  • Alzu’bi, A., Amira, A., & Ramzan, N. (2015). Semantic content-based image retrieval: A comprehensive study. Journal of Visual Communication and Image Representation, 32, 20–54. https://doi.org/10.1016/j.jvcir.2015.07.012
  • Alzu’bi, A., Amira, A., & Ramzan, N. (2017). Content-based image retrieval with compact deep convolutional features. In Neurocomputing (Vol. 249, pp. 95–105). https://doi.org/10.1016/j.neucom.2017.03.072
  • Ashraf, R., Ahmed, M., Jabbar, S., Khalid, S., Ahmad, A., Din, S., & Jeon, G. (2018). Content Based Image Retrieval by Using Color Descriptor and Discrete Wavelet Transform. Journal of Medical Systems, 42(3). https://doi.org/10.1007/s10916-017-0880-7
  • Aslandogan, Y. A., & Yu, C. T. (1999). Techniques and systems for image and video retrieval. In IEEE Transactions on Knowledge and Data Engineering (Vol. 11, Issue 1, pp. 56–63). https://doi.org/10.1109/69.755615
  • Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
  • Bouchakwa, M., Ayadi, Y., & Amous, I. (2020). Multi-level diversification approach of semantic-based image retrieval results. Progress in Artificial Intelligence, 9(1), 1–30. https://doi.org/10.1007/s13748-019-00195-x
  • Chen, H., Guo, A. Bin, Ni, W., & Cheng, Y. (2020). Improving the representation of image descriptions for semantic image retrieval with RDF. Journal of Visual Communication and Image Representation, 73(August 2019), 102934. https://doi.org/10.1016/j.jvcir.2020.102934
  • De Geus, D., Meletis, P., & Dubbelman, G. (2020). Fast panoptic segmentation network. IEEE Robotics and Automation Letters, 5(2), 1742–1749. https://doi.org/10.1109/LRA.2020.2969919
  • Deserno, T. M., Antani, S., & Long, R. (2009). Ontology of gaps in content-based image retrieval. In Journal of Digital Imaging (Vol. 22, Issue 2, pp. 202– 215). https://doi.org/10.1007/s10278-007-9092-x
  • Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016). Deep learning for visual understanding: A review. In Neurocomputing (Vol. 187, pp. 27–48). https://doi.org/10.1016/j.neucom.2015.09.116
  • Li, Y., Wang, Y., & Huang, X. (2007). A relation-based search engine in Semantic Web. IEEE Transactions on Knowledge and Data Engineering, 19(2), 273–281. https://doi.org/10.1109/TKDE.2007.18
  • Liu, Y., Zhang, D., Lu, G., & Ma, W. Y. (2007). A survey of content-based image retrieval with high-level semantics. Pattern Recognition, 40(1), 262–282. https://doi.org/10.1016/j.patcog.2006.04.045
  • Long, J., Shelhamer, E., & Darrell, T. (2015). Fully Convolutional Networks for Semantic Segmentation. In IEEE. https://doi.org/10.1109/CVPR.2015.7298965
  • Ma, H., Zhu, J., Lyu, M. R. T., & King, I. (2010). Bridging the semantic gap between image contents and tags. IEEE Transactions on Multimedia, 12(5), 462–473. https://doi.org/10.1109/TMM.2010.2051360
  • Mezaris, V., Kompatsiaris, I., & Strintzis, M. G. (2003). An ontology approach to object-based image retrieval. Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429), 3, II-511–514. https://doi.org/10.1109/ICIP.2003.1246729
  • Minu, R. I., & Thyagharajan, K. K. (2014). Semantic rule based image visual feature ontology creation. International Journal of Automation and Computing, 11(5), 489–499. https://doi.org/10.1007/s11633-014-0832-3
  • Ngo, T. G., Ngo, Q. T., & Nguyen, D. D. (2016). Image Retrieval with relevance feedback using SVM active learning. International Journal of Electrical and Computer Engineering, 6(6), 3238–3246. https://doi.org/10.11591/ijece.v6i6.11631
  • Noh, H., Hong, S., & Han, B. (2015). Learning Deconvolution Network for Semantic Segmentation (Vol. 1). https://doi.org/10.1109/ICCV.2015.178
  • Pang, Y., Li, Y., Shen, J., & Shao, L. (2019). Towards bridging semantic gap to improve semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, 2019-Octob(Iccv), 4229–4238. https://doi.org/10.1109/ICCV.2019.00433
  • Parsons, S. (2009). A Semantic Web Primer, Second Edition by Antoniou Grigoris and Harmelen Frank van, MIT Press, 288 pp.. In The Knowledge Engineering Review (Vol. 24, Issue 4). https://doi.org/10.1017/s0269888909990117
  • Rizwan I Haque, I., & Neubert, J. (2020). Deep learning approaches to biomedical image segmentation. In Informatics in Medicine Unlocked (Vol. 18). https://doi.org/10.1016/j.imu.2020.100297
  • Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000). Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1349–1380. https://doi.org/10.1109/34.895972
  • Song, K., Li, F., Long, F., Wang, J., & Ling, Q. (2018). Discriminative Deep Feature Learning for Semantic-Based Image Retrieval. IEEE Access, 6, 44268– 44280. https://doi.org/10.1109/ACCESS.2018.2862464
  • Tzelepi, M., & Tefas, A. (2018). Deep convolutional learning for Content Based Image Retrieval. In Neurocomputing (Vol. 275, pp. 2467–2478). https://doi.org/10.1016/j.neucom.2017.11.022
  • Wang, Q., Lai, J., Claesen, L., Yang, Z., Lei, L., & Liu, W. (2020). A novel feature representation: Aggregating convolution kernels for image retrieval. Neural Networks, 130, 1–10. https://doi.org/10.1016/j.neunet.2020.06.010
  • Wu, Q. (2020). Image retrieval method based on deep learning semantic feature extraction and regularization softmax. Multimedia Tools and Applications, 79(13–14), 9419–9433. https://doi.org/10.1007/s11042-019-7605-5
  • Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2020). Deep multimodal fusion for semantic image segmentation: A survey. Image and Vision Computing, 104042. https://doi.org/https://doi.org/10.1016/j.imavis.2020.104042
  • Zhao, R., & Grosky, W. I. (2002). Narrowing the semantic gap - Improved text-based web document retrieval using visual features. IEEE Transactions on Multimedia, 4(2), 189–200. https://doi.org/10.1109/TMM.2002.1017733
  • Zhu, H. (2020). Massive-scale image retrieval based on deep visual feature representation. Journal of Visual Communication and Image Representation, 70. https://doi.org/10.1016/j.jvcir.2019.102738
  • WordNet. Retrieved from https://wordnet.princeton.edu, (22.11.2020)
There are 32 citations in total.

Details

Primary Language Turkish
Subjects Computer Software
Journal Section Review
Authors

Akif Gaşi 0000-0001-8049-1273

Tolga Ensari 0000-0003-0896-3058

Mustafa Dağtekin 0000-0002-0797-9392

Early Pub Date September 13, 2021
Publication Date December 30, 2021
Submission Date December 11, 2020
Published in Issue Year 2021 Volume: 5 Issue: 2

Cite

APA Gaşi, A., Ensari, T., & Dağtekin, M. (2021). Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme. Acta Infologica, 5(2), 445-457.
AMA Gaşi A, Ensari T, Dağtekin M. Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme. ACIN. December 2021;5(2):445-457.
Chicago Gaşi, Akif, Tolga Ensari, and Mustafa Dağtekin. “Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme”. Acta Infologica 5, no. 2 (December 2021): 445-57.
EndNote Gaşi A, Ensari T, Dağtekin M (December 1, 2021) Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme. Acta Infologica 5 2 445–457.
IEEE A. Gaşi, T. Ensari, and M. Dağtekin, “Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme”, ACIN, vol. 5, no. 2, pp. 445–457, 2021.
ISNAD Gaşi, Akif et al. “Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme”. Acta Infologica 5/2 (December 2021), 445-457.
JAMA Gaşi A, Ensari T, Dağtekin M. Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme. ACIN. 2021;5:445–457.
MLA Gaşi, Akif et al. “Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme”. Acta Infologica, vol. 5, no. 2, 2021, pp. 445-57.
Vancouver Gaşi A, Ensari T, Dağtekin M. Anlamsal Tabanlı Görüntü Erişimi Üzerine Bir Derleme. ACIN. 2021;5(2):445-57.
 Download Cover Image