Speech Communication through the Skin: Design of Learning Protocols and Initial Findings

Abstract

Evidence for successful communication through the sense of touch is provided by the natural methods of tactual communication that have been used for many years by persons with profound auditory and visual impairments. However, there are still currently no wearable devices that can support speech communication effectively without extensive learning. The present study reports the design and testing of learning protocols with a system that translates English phonemes to haptic stimulation patterns (haptic symbols). In one pilot study and two experiments, six participants went through the learning and testing of phonemes and words that involved different vocabulary sizes and learning protocols. We found that with a distinctive set of haptic symbols, it was possible for the participants to learn phonemes and words in small chunks of time. Further, our results provided evidence of the memory consolidation theory in that recognition performance improved after a period of inactivity on the part of a participant. Our findings pave the way for future work on improving the haptic symbols and on protocols that support the learning of a tactile speech communication system in hours as opposed to the much longer periods of time that are required to learn a new language.