Automatic Variation of the Degree of Articulation in New HMM-based Voices

[en] This paper focuses on the automatic modification of the degree of articulation (hypo and hyperarticulation) of an existing standard neutral voice in the framework of HMM-based speech synthesis. Hypo and hyperarticulation refer to the production of speech respectively with a reduction and an increase of the articulatory efforts compared to the neutral style. Starting from a source speaker for which neutral, hypo and hyperarticulated speech data are available, statistical transformations are computed during the adaptation of the neutral speech synthesizer. These transformations are then applied to a new target speaker for which no hypo or hyperarticulated recordings are available. Four statistical methods are investigated, differing in the speaking style adaptation technique (model-space Linear Scaling LS vs. CMLLR) and in the speaking style transposition approach (phonetic vs. acoustic correspondence) they use. The efficiency of these techniques is assessed for the transposition of prosody and of filter coefficients separately. Besides we investigate which representation of the spectral envelope is the most suited for this purpose: MGC, LSP, PARCOR and LAR coefficients. Subjective evaluations are performed in order to determine which statistical transformation method achieves the highest performance in terms of segmental quality, reproduction of the articulation degree and speaker identity preservation. The most successful method is finally used for automatically modifying the degree of articulation of existing standard neutral voices.

Disciplines :

Electrical & electronics engineering

Author, co-author :

Picart, Benjamin ; Université de Mons > Faculté Polytechnique > Information, Signal et Intelligence artificielle

Drugman, Thomas ; Université de Mons > Faculté Polytechnique > Information, Signal et Intelligence artificielle

Dutoit, Thierry ; Université de Mons > Faculté Polytechnique > Information, Signal et Intelligence artificielle

Language :

English

Title :

Automatic Variation of the Degree of Articulation in New HMM-based Voices

Publication date :

11 March 2014

Journal title :

IEEE Journal of Selected Topics in Signal Processing

ISSN :

1932-4553

Publisher :

Institute of Electrical and Electronics Engineers, New York, United States - New York

Pages :

307 - 322

Peer reviewed :

Peer Reviewed verified by ORBi

Research unit :

F105 - Information, Signal et Intelligence artificielle

Research institute :

R450 - Institut NUMEDIART pour les Technologies des Arts Numériques

Available on ORBi UMONS :

since 23 January 2014

Statistics

Number of views

3 (0 by UMONS)

Number of downloads

1 (1 by UMONS)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

B. Lindblom, Economy of Speech Gestures. New York, NY, USA: Spinger-Verlag, 1983.
G. Beller, "Analyse et modèle génératif de l'expressivité-Application à la parole et à l'interprétation musicale," Ph.D. dissertation, Université Paris VI-Pierre et Marie Curie, IRCAM, Paris, France, 2009.
G. Beller, "Influence de l'expressivité sur le degré d'articulation," in Proc. RJC Parole, France, Jul. 2007.
G. Beller, N. Obin, and X. Rodet, "Articulation degree as a prosodic dimension of expressive speech," in Proc. 4th Int. Conf. Speech Prosody, Campinas, Brazil, 2008.
B. Picart, T. Drugman, and T. Dutoit, "Analysis and synthesis of hypo and hyperarticulated speech," in Speech Synthesis Workshop 7 (SSW7), Kyoto, Japan, Sep. 22-24, 2010, pp. 270-275.
B. Picart, T. Drugman, and T. Dutoit, "Analysis and HMM-based synthesis of hypo and hyperarticulated speech," Comput. Speech Lang., Special Iss. The Listening Talker, vol. 28, no. 2, pp. 687-707, Mar. 2014.
J. Laver, Principles of Phonetics. Great Britain: Cambridge Textbooks in Linguistics. Cambridge, U.K.: Cambridge Univ. Press, 1994.
E. Lombard, "Le signe de l'élévation de la voix," Ann. Maladies de l'Oreille et du Larynx, vol. 37, no. 2, pp. 101-119, 1911.
W. V. Summers, D. Pisoni, R. Bernacki, R. Pedlow, and M. Stokes, "Effects of noise on speech production: Acoustic and perceptual analyses," J. Acoust. Soc. Amer. (JASA), vol. 84, no. 3, pp. 917-928, Sep. 1988.
M. Garnier, L. Bailly, M. Dohen, P. Welby, and H. Loevenbruck, "The Lombard effect: A physiological reflex or a controlled intelligibility enhancement?," in Proc. Int. Seminar on Speech Production (ISSP), Ubatuba, Brazil, Dec. 2006, pp. 255-262.
M. Garnier, L. Bailly, M. D. P. Welby, and H. Loevenbruck, "An acoustic and articulatory study of Lombard speech. Global effects at utterance level," in Proc. Int. Conf. Spoken Lang. Process. (ICSLP), Pittsburgh, PA, USA, Sep. 2006, pp. 2246-2249.
T. Drugman and T. Dutoit, "Glottal-based analysis of the Lombard effect," in Proc. Interspeech, Makuhari, Japan, 2010, pp. 2610-2613.
Y. Lu and M. Cooke, "The contribution of changes in F0 and spectral tilt to increased intelligibility of speech produced in noise," Speech Commun., vol. 51, no. 12, pp. 1253-1262, Dec. 2009.
J. Yamagishi, T. Nose, H. Zen, Z. Ling, T. Toda, K. Tokuda, S. King, and S. Renals, "A robust speaker-adaptive HMM-based text-to-speech synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 6, pp. 1208-1230, Aug. 2009.
T. Nose, M. Tachibana, and T. Kobayashi, "HMM-based style control for expressive speech synthesis with arbitrary speaker's voice using model adaptation," IEICE Trans. Inf. Syst., vol. 92, no. 3, pp. 489-497, 2009.
T. Toda, Y. Ohtani, and K. Shikano, "Eigenvoice conversion based on Gaussian mixture model," in Proc. Interspeech, Pittsburgh, PA, USA, Sep. 2006, pp. 2446-2449.
Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano, "Adaptive training for voice conversion based on eigenvoices," IEICE Trans. Inf. Syst., vol. 93, no. 6, pp. 1589-1598, Jun. 2010.
Y. Stylianou, O. Cappe, and E. Moulines, "Continuous probabilistic transform for voice conversion," IEEE Trans. Speech Audio Process., vol. 6, no. 2, pp. 131-142, Mar. 1998. (Pubitemid 128720639)
H. Ye and S. Young, "High quality voice morphing," in Proc. IEEE ICASSP, Montreal, QC, Canada, 2004, pp. 9-12.
M. Nicolao, J. Latorre, and R. K. Moore, "C2H: A computational model of H&H-based phonetic contrast in synthetic speech," in Interspeech, Portland, OR, USA, 2012.
R. K. Moore and M. Nicolao, "Reactive speech synthesis: Actively managing phonetic contrast along an H&H continuum," in Proc. 17th Int. Congr. Phon. Sci. (ICPhS), Hong Kong, 2011.
M. Nicolao, F. Tesser, and R. K. Moore, "A phonetic-contrast motivated adaptation to control the degree-of-articulation on italian HMM-based synthetic voices," in Proc. Speech Synth. Workshop 8 (SSW8), Barcelona, Spain, Aug. 31-Sep. 2 2013, pp. 107-112.
H. Kanagawa, T. Nose, and T. Kobayashi, "Speaker-independent style conversion for HMM-based expressive speech synthesis," in IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Vancouver, BC, Canada, May 26-31, 2013, pp. 7864-7868.
M. Tachibana, J. Yamagishi, T. Masuko, and T. Kobayashi, "A style adaptation technique for speech synthesis using HSMM and suprasegmental features," IEICE Trans. Inf. Syst., vol. 89, no. 3, pp. 1092-1099, 2006.
J. Yamagishi, "Average-voice-based speech synthesis," Ph.D. dissertation, Tokyo Inst. of Technol., Tokyo, Japan, Mar. 2006.
B. Picart, T. Drugman, and T. Dutoit, "Continuous control of the degree of articulation in HMM-based speech synthesis," in Proc. Interspeech, Florence, Italy, Aug. 27-31, 2011, pp. 1797-1800.
B. Picart, T. Drugman, and T. Dutoit, "Perceptual effects of the degree of articulation in HMM-based speech synthesis," in Proc. NOLISP Workshop, Las Palmas, Gran Canaria, 2011, pp. 177-182.
B. Picart, T. Drugman, and T. Dutoit, "HMM-based speech synthesis with various degrees of articulation: A perceptual study," Neurocomputing J., Special Iss. NOLISP '11, 2013, to be published.
B. Picart, T. Drugman, and T. Dutoit, "Assessing the intelligibility and quality of HMM-based speech synthesis with a variable degree of articulation," in Proc. The Listening Talker (LISTA) Workshop, Edinburgh, U.K., May 2012.
B. Picart, T. Drugman, and T. Dutoit, "Statistical methods for varying the degree of articulation in new HMM-based voices," in IEEE Workshop on Spoken Language Technology (SLT), Miami, FL, USA, Dec. 2-5, 2012, pp. 291-296.
K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, "Mel-generalized cepstral analysis-A unified approach to speech spectral estimation," in Proc. Int. Conf. Spoken Lang. Process., Sep. 1994, vol. 3, pp. 1043-1046.
F. Itakura, "Line spectrum representation of linear predictor coefficients of speech signals," J. Acoust. Soc. Amer. (JASA), vol. 57, p. S35, 1975.
T. Dutoit, An Introduction to Text-to-Speech Synthesis. Norwell, MA, USA: Kluwer, 1997.
R. Boite, H. Bourlard, T. Dutoit, J. Hancq, and H. Leich, Traitement de la Parole. Lausanne, Switzerland: Presses Polytechniques et Universitaires Romandes, 1999.
L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition, ser. Signal Processing. Englewood Cliffs, NJ, USA: Prentice Hall, 1993.
K. K. Paliwal, "Interpolation properties of linear prediction parametric representations," in Proc. Eurospeech, Madrid, Spain, Sep. 18-21, 1995, pp. 1029-1032.
T. Bäckström and C. Magi, "Properties of line spectrum pair polyno-mials-A review," Signal Process., vol. 86, no. 11, pp. 3286-3298, 2006. (Pubitemid 44331418)
V. Hazan and R. Baker, "Does reading clearly produce the same acoustic-phonetic modifications as spontaneous speech in a clear speaking style?," in Proc. DiSS-LPSS Joint Workshop, 2010, pp. 7-10.
S. H. Ferguson, "Vowels in clear and conversational speech: Talker differences in acoustic features and intelligibility for normal-hearing listeners," Ph.D. dissertation, Indiana Univ., Bloomington, IN, USA, 2002.
R. E. Baker and A. R. Bradlow, "Variability in word duration as a function of probability, speech style, and prosody," Lang. Speech, vol. 52, no. 4, pp. 391-413, Dec. 2009.
M. P. Aylett, "Stochastic suprasegmentals: Relationships between redundancy, prosodic structure and care of articulation in spontaneous speech," Ph.D. dissertation, Univ. of Edinburgh, Edinburgh, U.K., 2000.
J. Brink, R. Wright, and D. B. Pisoni, Eliciting speech reduction in the laboratory: Assessment of a new experimental method Speech Research Lab., Dept.of Psychol., Indiana Univ., Bloomington, IN, USA, 1998, Tech. Rep.
J. D. Harnsberger, R. Wright, and D. B. Pisoni, "A new method for eliciting three speaking styles in the laboratory," Speech Commun., vol. 50, no. 4, pp. 323-336, 2008.
B. Langner and A. W. Black, "Creating a database of speech in noise for unit selection synthesis," in Proc. Speech Synth. Workshop 5 (SSW5), Pittsburgh, PA, USA, 2004, pp. 229-230.
F. R. Chen, "Acoustic characteristics and intelligibility of clear and conversational speech at the segmental level," M.S. thesis, Dept. of Elect. Eng. and Comput. Sci., Mass. Insti. of Technol., Cambridge, MA, USA, 1980.
K. Sjolander and J. Beskow, "Wavesurfer-An open source speech tool," in Proc. Int. Conf. Spoken Lang. Process. (ICSLP), 2000, vol. 4, pp. 464-467.
H. Zen, K. Tokuda, and A. W. Black, "Statistical parametric speech synthesis," Speech Commun., vol. 51, no. 11, pp. 1039-1064, 2009.
[Online]. Available: http://hts.sp.nitech.ac.jp/HMM-Based Speech Synthesis System (HTS) [Online]. Available: http://hts.sp.nitech.ac.jp/
T. Drugman and T. Dutoit, "The deterministic plus stochastic model of the residual signal and its applications," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 3, pp. 968-981, Mar. 2012.
V. Digalakis, D. Rtischev, and L. Neumeyer, "Speaker adaptation using constrained reestimation of gaussian mixtures," IEEE Trans. Speech Audio Process., vol. 3, no. 5, pp. 357-366, Sep. 1995.
M. Gales, "Maximum likelihood linear transformations for HMM-based speech recognition," Comput. Speech Lang., vol. 12, no. 2, pp. 75-98, Apr. 1998. (Pubitemid 128383747)
J. Ferguson, "Variable duration models for speech," in Proc. Symp. Applicat. Hidden Markov Models to Text and Speech, 1980, pp. 143-179.
H. Zen, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "A hidden semi-markov model-based speech synthesis system," IEICE Trans. Inf. Syst., vol. 90, no. 5, pp. 825-834, May 2007.
T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kita-mura, "Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis," in Proc. Eurospeech, Sep. 1999, pp. 2347-2350.
J. Yamagishi, "Average-Voice-Based Speech Synthesis," Ph.D. dissertation, Tokyo Inst. of Technol., Tokyo, Japan, Mar. 2006.
H. Liang, J. Dines, and L. Saheer, "A comparison of supervised and unsupervised cross-lingual speaker adaptation approaches for HMM-based speech synthesis," in Proc. IEEE ICASSP, Dallas, TX, 2010, pp. 4598-4601.
D. C. Howell, Statistical Methods for Psychology. Independence, KY, USA: Wadsworth, 2012.