Real-Time Recognition of American Sign Language Using Complex Zernike Moments

Bayrak S., Nabiyev V., Atalar C.

6TH INTERNATIONAL CONFERENCE ON APPLIED ENGINEERING AND NATURAL SCIENCES ICAENS 2024, Konya, Türkiye, 25 Eylül - 26 Ekim 2024, ss.560-565

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Basıldığı Şehir: Konya
  • Basıldığı Ülke: Türkiye
  • Sayfa Sayıları: ss.560-565
  • Karadeniz Teknik Üniversitesi Adresli: Evet

Özet

This study addresses the recognition of American Sign Language (ASL) using a real-time system built on the MUD dataset, which includes 36 sign classes representing letters and digits. The system employs complex Zernike moments for feature extraction and utilizes a complex-valued deep neural network to classify hand signs captured via webcam. In the study, the Mediapipe library, which identifies key points on the hand, was used to detect the hand and to ensure consistent feature extraction, the hand was framed in a square. Initial results demonstrate a recognition accuracy between 90% and 95%, though challenges remain, particularly in distinguishing similar signs. The study emphasizes the impact of background and lighting on performance, suggesting that further optimization and training with lower-resolution images could enhance speed and efficiency, ultimately supporting mobile applications. The findings indicate that effective real-time sign language recognition systems require robust models and careful environmental control.