The development of an augmented reality audio application for visually impaired persons

AKIN A. T., CÖMERT Ç.

MULTIMEDIA TOOLS AND APPLICATIONS, cilt.82, sa.11, ss.17493-17512, 2023 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 82 Sayı: 11
  • Basım Tarihi: 2023
  • Doi Numarası: 10.1007/s11042-022-14134-x
  • Dergi Adı: MULTIMEDIA TOOLS AND APPLICATIONS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, FRANCIS, ABI/INFORM, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC, zbMATH
  • Sayfa Sayıları: ss.17493-17512
  • Anahtar Kelimeler: Augmented reality, Object detection, Navigation, Monocular depth extraction, NAVIGATION
  • Karadeniz Teknik Üniversitesi Adresli: Evet

Özet

In this study, an augmented reality audio application that works with smartphones has been developed to assist the lives of visually impaired persons. The application provides object detection, obstacle notification, and navigation through online base maps with audio feedback. Several important issues were to be tackled in such an undertaking. Deep learning techniques have been employed for the issues of monocular depth extraction and object detection. A web services solution has been adopted concerning real-time feedback, which is critical for the impaired. A deep learning monocular depth extraction model, which has been preferred with respect to a literature review, has been validated with relevant metrics. For object detection, a well-proven and widely used deep learning model has been chosen. All the involved software components and the developed application are open source.