Lip-Geometry Feature-Based Visual Digit Recognition

Abstract

One of the most prevalent and natural forms of expression is speech. The level of speech understanding can be increased by the human using visual clues like lip and tongue movements. Visual speech recognition is the act of understanding speech by seeing the speaker’s motion of the lips. To recognize visual speech, shape or geometry-based characteristics extract the speaker’s lip movement. This paper proposes a lip geometry-based visual feature for visual digit recognition. The lip geometry of a speaker is calculated using the Pseudo-Zernike Moment Invariant (PZMI). Artificial Neural Network (ANN) and Radial Basis Function Neural Network (RBF-NN) are used here to recognize the speech for visual modality. The aim of the proposed work is to extract translation, rotation, and scale-invariant visual features. Moments invariant features are used in classification and recognition work as pattern-sensitive features. These features are proving discriminating properties for similar images which is very important for recognizing different visual speech. Because the accuracy of these futures has a significant impact on the classifiers used. The proposed system achieves 80% and 78.3% recognition accuracy using RBF-NN and ANN, respectively. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.