Copyright © 2008 The Institute of Electronics, Information and Communication Engineers
Special Section on Robust Speech Processing in Realistic Environments -- Papers -- ASR under Reverberant Conditions |
Recognizing Reverberant Speech Based on Amplitude and Frequency Modulation
1 The authors are with the School of Science and Engineering, Waseda University, Tokyo, 169–8555 Japan. E-mail: yotaro{at}shirai.cs.waseda.ac.jp, 2 The author is with the Chiba Institute of Technology, Narashino-shi, 275–0061 Japan.
We have attempted to recognize reverberant speech using a novel speech recognition system that depends on not only the spectral envelope and amplitude modulation but also frequency modulation. Most of the features used by modern speech recognition systems, such as MFCC, PLP, and TRAPS, are derived from the energy envelopes of narrowband signals by discarding the information in the carrier signals. However, some experiments show that apart from the spectral/time envelope and its modulation, the information on the zero-crossing points of the carrier signals also plays a significant role in human speech recognition. In realistic environments, a feature that depends on the limited properties of the signal may easily be corrupted. In order to utilize an automatic speech recognizer in an unknown environment, using the information obtained from other signal properties and combining them is important to minimize the effects of the environment. In this paper, we propose a method to analyze carrier signals that are discarded in most of the speech recognition systems. Our system consists of two nonlinear discriminant analyzers that use multilayer perceptrons. One of the nonlinear discriminant analyzers is HATS, which can capture the amplitude modulation of narrowband signals efficiently. The other nonlinear discriminant analyzer is a pseudo-instantaneous frequency analyzer proposed in this paper. This analyzer can capture the frequency modulation of narrowband signals efficiently. The combination of these two analyzers is performed by the method based on the entropy of the feature introduced by Okawa et al. In this paper, in Sect.2, we first introduce pseudo-instantaneous frequencies to capture a property of the carrier signal. The previous AM analysis method are described in Sect.3. The proposed system is described in Sect.4. The experimental setup is presented in Sect.5, and the results are discussed in Sect.6. We evaluate the performance of the proposed method by continuous digit recognition of reverberant speech. The proposed system exhibits considerable improvement with regard to the MFCC feature extraction system.
Key Words: speech recognition, temporal feature, tandem approach, multistream combination, reverberant speech
Manuscript received June 28, 2007. Manuscript revised September 12, 2007.
Reference
[1] H. Hermansky, "Perceptual linear predictive (PLP) analysis of speech," J. Acousti. Soc. Am., vol.87, pp.1738–1752, April 1990. [2] H. Hermansky and S. Sharma, "TRAPS — Classifiers of temporal patterns," Proc. ICSLP'98, Sydney, Australia, Nov. 1998. [3] K. Yoshida, M. Kazama, and M. Tohyama, "Pitch and speech-rate conversion using envelope modulation modeling," Proc. ICASSP-2002, I.435–I.428, Orland, 2002. [4] H. Hermansky, "Should recognizers have ears?," Speech Commun., vol.25, no.1–3, pp.3–27, 1998. [5] H. Hermansky and N. Morgan, "RASTA processing of speech," IEEE Trans. Speech Audio Process., vol.2, no.4, pp.578–589, Oct. 1994. [6] T. Houtgast and H.J.M. Steeneken, "A review of the MTF concept in room acoustics and its use for estimating speech intelligibility in auditoria," J. Acoust. Soc. Am., vol.77, pp.1069–1077, March 1985. [7] Y. Wang, J. Hansen, G.K. Allu, and R. Kumaresan, "Average Instantaneous Frequency (AIF) and Average Log-Envelopes (ALE) for ASR with the aurora 2 database," Proc. Eurospeech 2003, pp.25–28, 2003. [8] D. Dimitriadis, P. Maragos, and A. Potamianos, "Robust AM-FM features for speech recognition," IEEE Signal Process. Lett., vol.12, no.9, pp.621–624, Sept. 2005. [9] J. Bilmes, "Maximum mutual information based reduction strategies for cross correlation based joint distributional modeling," Proc. ICASSP-98, pp.469–472, Seattle, May 1998. [10] N. Kanedera, T. Arai, and T. Funada, "Robust automatic speech recognition emphasizing important modulation spectrum," IEICE Trans. Inf. & Syst. (Japanese Edition), vol.J84-D-II, no.7, pp.1261–1269, July 2001. [11] N. Morgan, Q. Zhu, A. Stolcke, K. Sönmez, S. Sivadas, T. Shinozaki, M. Ostendorf, P. Jain, H. Hermansky, D. Ellis, G. Doddington, B. Chen, Ö. Çetin, H. Bourlard, and M. Athineos, "Pushing the envelope – Aside," IEEE Signal Process. Mag., vol.22, no.5, pp.81–88, Sept. 2005. [12] B. Chen, S. Chang, and S. Sivadas, "Learning discriminative temporal patterns in speech: Development of novel TRAPS-like classifiers," Proc. Eurospeech, pp.429–432, Geneve, 2003. [13] B. Chen, S. Chang, and S. Sivadas, "Learning long term temporal features in LVCSR using neural networks," Proc. INTERSPEECH-ICSLP-2004, pp.612–615, Jeju Island, Korea, Oct. 2004. [14] S. Okawa, E. Bocchieri, and A. Potamianos, "Multi-band speech recognition in noisy environments," Proc. ICASSP-98, pp.641–644, Seattle, Washington, USA, May 1998. [15] H. Suzuki, F. Ma, H. Izumi, O. Yamazaki, S. Okawa, and K. Kido, "Instantaneous frequencies of signals obtained by the analytic signal," Acoustical Science & Technology, vol.27, no.3, pp.163–170, May 2006. [16] H. Hermansky, D.P.W. Ellis, and S. Sharma, "Tandem connectionist feature extraction for conventional HMM systems," Proc. ICASSP-2000, pp.1635–1638, Istanbul, June 2000. [17] S. Ikbal, H. Misra, S. Sivadas, H. Hermansky, and H. Bourlard, "Entropy based combination of tandem representations for noise robust ASR," Proc. INTERSPEECH-ICSLP-2004, pp.2553–2556, Jeju Island, Korea, Oct. 2004. [18] CENSREC-1: http://sp.shinshu-u.ac.jp/CENSREC/ja/CENSREC/AURORA-2J/ [19] E. Zwicker, "Die Grenzen der Hoerbarkeit der Amplitudenmodulation under der Frequenzmodulation eines Tones," Acusstica, vol.2, pp.125–133, 1952. [20] G. Green, Temporal Aspects of Audition, Ph.D. Thesis, Oxford University, 1976.
CiteULike 
Connotea 
Del.icio.us What's this?
This Article 
Abstract
Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Services
Email this article to a friend
Similar articles in this journal
Alert me to new issues of the journal
Add to My Personal Archive
Download to citation manager
Request Permissions
Google Scholar
Articles by KUBO, Y.
Articles by SHIRAI, K.
Social Bookmarking
What's this?