MULTIMODAL SYSTEM FOR HANDS-FREE PC CONTROL (ThuPmOR11)

Author(s) : Andrey Ronzhin (St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS), Russian Federation) Alexey Karpov (St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS), Russian Federation) Alexander Nechaev (St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS), Russian Federation) Svetlana Chernakova (St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences (SPIIRAS), Russian Federation)

Abstract : This paper describes the developed multimodal system intended for assistance to people with disabilities of hands. It combines automatic speech recognition and head tracking in one multimodal system. The structure of the system, the methods for recognition and tracking, information fusion and synchronization, the obtained results and testing conditions are described in the paper. This system was applied for hands-free control of Graphical User Interface for such tasks as Internet communication and text editing in MS Word.

Menu