The development of an augmented reality audio application for visually impaired persons

AKIN A. T., CÖMERT Ç.

MULTIMEDIA TOOLS AND APPLICATIONS, vol.82, no.11, pp.17493-17512, 2023 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 82 Issue: 11
  • Publication Date: 2023
  • Doi Number: 10.1007/s11042-022-14134-x
  • Journal Name: MULTIMEDIA TOOLS AND APPLICATIONS
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, FRANCIS, ABI/INFORM, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC, zbMATH
  • Page Numbers: pp.17493-17512
  • Keywords: Augmented reality, Object detection, Navigation, Monocular depth extraction, NAVIGATION
  • Karadeniz Technical University Affiliated: Yes

Abstract

In this study, an augmented reality audio application that works with smartphones has been developed to assist the lives of visually impaired persons. The application provides object detection, obstacle notification, and navigation through online base maps with audio feedback. Several important issues were to be tackled in such an undertaking. Deep learning techniques have been employed for the issues of monocular depth extraction and object detection. A web services solution has been adopted concerning real-time feedback, which is critical for the impaired. A deep learning monocular depth extraction model, which has been preferred with respect to a literature review, has been validated with relevant metrics. For object detection, a well-proven and widely used deep learning model has been chosen. All the involved software components and the developed application are open source.