Authors:
Jose A. Gonzalez
1
;
Lam A. Cheah
2
;
James M. Gilbert
2
;
Jie Bai
2
;
Stephen R. Ell
3
;
Phil D. Green
1
and
Roger K. Moore
1
Affiliations:
1
University of Sheffield, United Kingdom
;
2
University of Hull, United Kingdom
;
3
Hull and East Yorkshire Hospitals Trust, United Kingdom
Keyword(s):
silent speech interfaces, speech rehabilitation, speech synthesis and permanent magnet articulography
Related
Ontology
Subjects/Areas/Topics:
Acoustic Signal Processing
;
Artificial Intelligence
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Data Manipulation
;
Devices
;
Electromagnetic Fields in Biology and Medicine
;
Health Engineering and Technology Applications
;
Health Information Systems
;
Human-Computer Interaction
;
Methodologies and Methods
;
Multimedia
;
Multimedia Signal Processing
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Soft Computing
;
Speech Recognition
;
Telecommunications
;
Wearable Sensors and Systems
Abstract:
Patients with larynx cancer often lose their voice following total laryngectomy. Current methods for post-laryngectomy voice restoration are all unsatisfactory due to different reasons: requires frequent replacement due to biofilm growth (tracheo-oesoephageal valve), speech sounds gruff and masculine (oesophageal speech) or robotic (electro-larynx) and, in general, are difficult to master (oesophageal speech and electro-larynx). In this work we investigate an alternative approach for voice restoration in which speech articulator movement is converted into audible speech using a speaker-dependent transformation learned from simultaneous recordings of articulatory and audio signals. To capture articulator movement, small magnets are attached to the speech articulators and the magnetic field generated while the user `mouths' words is captured by a set of sensors. Parallel data comprising articulatory and acoustic signals recorded before laryngectomy are used to learn the mapping betwee
n the articulatory and acoustic domains, which is represented in this work as a mixture of factor analysers. After laryngectomy, the learned transformation is used to restore the patient's voice by transforming the captured articulator movement into an audible speech signal. Results reported for normal speakers show that the proposed system is very promising.
(More)