 Open Access
 Authors : Priyanka T B, Sindhu, Uma D, Varshitha S M, Manjula G
 Paper ID : IJERTCONV9IS12008
 Volume & Issue : NCCDS – 2021 (Volume 09 – Issue 12)
 Published (First Online): 20072021
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
Speech Enhancement using Hidden Markov Models and MelFrequency
Priyanka T B1, Sindhu2, Uma D3, Varshitha S M4, Manjula G5
B.E Students1234 (ECE) at GSSS Institute of Engineering & Technology for women, Mysore, Karnataka, India. Asst.prof.5Dept of ECE, GSSSIETW, Mysore, Karnataka, India.
Visvesvaraya Technology University, Belagavi, Karnataka, India
Abstract – Speech enhancement using Hidden Markov model (HMM) – based minimum mean square error in Melfrequency domain is mainly focused and to estimate clean speech waveform from a noisy signal, an inversion from the Mel frequency domain to the spectral domain is required which introduces distortion artifacts in the spectrum estimation and filtering. To obtain a more accurate hidden Markov model (HMM) of noisy speech using the Vector Taylor Series (VTS) which is used to estimate the mean vectors and covariance matrices of HMM for noisy speech. To reduce the distortion derived from the inversion operation, a parallel Melfrequency and log magnitude (PMLM) modeling is used. Experimental results show that, in comparison with the reference methods, the proposed method can get better performance for different noise environments and input SNRs.
Keywords – HMMbased speech enhancement, vector Taylor series, Parallel Melfrequency and log magnitude modeling.
INTRODUCTION
Speech signals need to be enhanced in many applications for various purposes such as boosting overall speech quality, increasing intelligibility or improving the performance of speech coding and speech recognition systems. Although speech enhancement has been studied in various aspects, single microphone speech enhancement is of wide interest due to the extent and variety of its applications. Furthermore, it has remained as a challenging topic over the years and numerous researchers have tried to suggest solutions for this problem.
Speech enhancement using HMM is originally proposed by Ephraim [1], then the speech enhancement based on minimum mean square error (MMSE) criterion using HMM is proposed [2]. Later, some scholars made many improvements based on HMM [35], including the changes of the training features. ZhenZhen gao [6] discuss the HMMbased speech enhancement using VTS and PMLM modeling in Me frequency domain. The VTS is used to estimate the mean vectors and covariance matrices of noisy speech HMM more accurately, and the PMLM modeling is used to reduce the spectral distortion resulted by the inversion from Mel frequency domain to spectral domain.
Speech enhancement aims to improve speech quality by using various algorithms. The objective of enhancement is improvement in intelligibility and/or overall perceptual quality
of degraded speech signal using audio signal processing techniques. The algorithms of speech enhancement for noise reduction can be categorized into three fundamental classes: filtering techniques, spectral restoration, and modelbased methods. This project uses filtering method.
Speech enhancement is required in many applications such cellular phones, handsfree communication, teleconferencing, hearing aids, speech recognition and audio fingerprinting among others. If the speech signal is corrupted by additive noise the goal of speech enhancement is to find an optimal estimate ~s(n) of the clean speech signal s(n) , given a noisy observation x(n):
x(n) = s(n) + v(n) where v(n) is the additive noise Conventional speech enhancement methods, such as spectral subtraction, wiener filter, shorttime spectral amplitude estimator rely on a separate noise estimation algorithm, while the noise estimation is not accurate under nonstationary conditions. However, the speech enhancement based on hidden Markov model (HMM) can overcome the deficiencies. Hidden Markov Model (HMM) of noisy speech and each individual filter is estimated using the HMM of clean speech and noise. So, the more accurate the HMM of noisy speech and these filters are estimated, the better performance will be obtained.
METHODOLOGY
From Fig. 1, we can see that,
The proposed speech enhancement method contains two processes: training process and enhancement process. In the training process, the HMMs of clean speech and noise are trained using PMLM modeling method, and the HMM of noisy speech is constructed by VTS. In the enhancement process, the wiener filter is estimated using the HMMs and noisy speech. estimate clean speech waveform from a noisy signal, an inversion from the Melfrequency domain to the spectral domain is required which introduces distortion artifacts in the spectrum estimation and the filtering. To reduce the corrupting effects of the inversion, the PMLM modeling is proposed. This method performs concurrent modeling in both cepstral and magnitude spectral domains.
Fig1:The block diagram of the proposed speech enhancement method
HMMbased speech enhancement in mfs domain
The noisy speech vector y in the nth frame can be expressed as follows:
y=s+d (1)
where s and d denote the clean speech and additive noise vectors in time domain, respectively. s, d and y are defined as the MFS feature vectors of clean speech, noise and noisy speech in the nth frame, respectively. A general MFS feature vector x is defined as:
log=(MEL(DFT(win(x)))) (2)
where the symbol, win, DFT,., MEL and log denote the operations of windowing, discrete Fourier transform, magnitude calculation, Melfiltering, logarithmic, respectively. Similar to x which denotes the MFS domain feature vector for x, the superscript notations spc, mag, logmag, and Mel indicate the DFT, magnitude of DFT, logmagnitude spectra and MEL coefficients of speech signal in the following parts, respectively. The HMMs of clean speech, noise and noisy speech in MFS domain are denoted as s, d and y, respectively. In this paper, it is assumed that the feature vectors of clean speech, noise and noisy speech in MFS domain have a
Gaussian distribution, and each HMM is defined as
= { N M a c } (3)
where N and M are the number of states and mixtures, respectively, ={} is the set of initial state probabilities, a={a} is the set of state transition probabilities, c={c } is the set of mixture weights, Âµ={Âµ()} and ={()} are the sets of Gaussian density parameters for mean vectors and covariance matrices, respectively, and the ranges for and
are defined as 1 N and 1M, respectively. To estimate the parameters of s and d, the standard maximum likelihood
the MMSE criterion is used to estimate clean speech s^spc
.
.
(4)
where Ns and Ms are the number of states and mixtures of clean speech HMM, respectively. Nd and Md are the number of states and mixtures of noise HMM, respectively. y P (ÃŸs, s, ÃŸd, dy0:n mfs) is the conditional probability of clean speech state s and mixture s, and noise state d and mixture
d given the noisy speech y0:n mfs. E(sspcy spc, ÃŸs, s, ÃŸd, d) is the conditional expectation of clean speech.
Weiner Filtering
SNR estimation
SNR estimation
Weiner filter
Weiner filter
Pitch synchronous of STFT
Pitch synchronous of STFT
Phase reconstruction
Phase reconstruction
Inverse STFT
Inverse STFT
For discretetime signals, the Wiener filter is a linear shift invariant (LSI) filter whose output is an estimate of the desired signal when its input is the noisy signal x(n). It minimizes the mean squared error (MSE) between the estimated signal and the desired signal.
F0 detection
F0 detection
Cmputation of window size
Computation of window size
Fig 2: Block diagram of Weiner filtering
In Eq. 4, the conditional probability is calculated by the forwardbackward algorithm. Due to the Gaussian mixtures, the conditional expectation can be obtained by the following Wiener filtering
E( s spc y spc , , )
n n s s, d, d
hidden Markov model with Baum reestimation is used.
The type of the HMM in this paper is ergodic and the transition from one state to another is assumed to have first order Markovian property. Given the noisy speech y:=[y, y,,y],
= H( s d s d). y spc (5)
n
n
Where
mag 2
mag 2
H( s d s d) = (s (ss))
( s(s+s))2 +(d (d+d))2 (6)
If the impulse response of the Wiener filter is h[n], then:
(7)
The MSE is
(8)
In order to minimize the MSE, we differentiate it with respect to h[i] where i = 0B. The result is the following set of equations:
(9)
where Rsx(i) is the crosscorrelation between s(n) and x(n) at lag i .These equations are called the WienerHopf equations.
If the filter is not restricted to be causal, we can obtain the transfer function by taking the DTFT of equation and rearranging the terms.
(10)
If we assume that the underlying speech signal and the noise Are uncorrelated, then
(11)
Hence we get
(12)
(13)
Parallel frequency and log – magnitude method
In this paper, the feature vectors of MFS domain obey Gaussian distribution, and taking into account the linear relationship between the feature vectors of MFS domain and the feature vectors of LOGMAG domain, so the feature vectors of LOGMAG domain should also obey Gaussian distribution. In order to not only utilize the benefits of the feature vectors of MFS domain in the
Fig3:Training process of parallel Mel frequency and log magnitude modeling
From Fig. 3 we can see that, the HMM in MFS domain is trained using the conventional HMM formulation given , and only mean vectors and covariance matrices of the HMM in LOGMAG domain are obtained. In order to create the concurrent HMM in LOGMAG domain, it is assumed that the alignment of the frames and states in the MFS and LOGMAG domains are identical. The mean vectors and covariance matrices of HMM in LOGMAG domain are given by Eq. 14 and Eq. 15 respectively, where xn logmag is the observation vector in the LOGMAG domain corresponding to xn mfs, n mfs() is the probability of being in mixture of state given the observation sequence x0:n mfs in the MFS domain, N indicates the total number of frames.
mfs
mfs
n=1 n (,).xn
n=1 n (,).xn
logmag(,) = mfs logmag
n=1 n (,) (14) logmag(,) =n=1 n (,) B.B
mfs
mfs
mfs T
mfs T
n=1 n (,) (15)
Where
B= xn logmag – logmag (,) (16)
Now, the mean vectors and covariance matrices of HMM in LOGMAG domain are known, the mean vectors in MAG domain can be obtained by
mag=exp( logmag+ logmag/2) (17)
HMMbased speech enhancement system but also eliminate i
the distortion effect of the inversion process in calculating the filters, we propose a parallel Melfrequency and log magnitude (PMLM) modeling approach to train the HMMs,
i ii
RESULTS
as shown in Fig. 3.
In this section we will compare the results of the input speech signal and after applying HMM algorithm. The original Speech of the signal and the HMM approaches are shown.
The results have been obtained in the form of various performance parameters. At very first the original signal has been obtained both before and after enhancement. The signal
enhancement is the technique which is used to improve the quality of the speech signal. The speech signal enhancement has been performed by using the of the hidden Markov model. The PMLM method has returned the feature extracted from the speech signal which is further used as the core sample to improve the quality of the speech signal using the HMM model. The speech signal has been obtained before and after the enhancement. There are the minute visible changes in the signal after the speech signal enhancement and so in the case of signal noise. The signal noise has been also calculated before and after the speech signal enhancement.
Noisy Speech is enhanced and noiseless speech is generated by filtering. Obtained a more accurate hidden Markov models (HMM) of noisy speech using the vector Taylor series (VTS) which is used to estimate the mean vectors and covariance matrices of HMM for noisy speech.
Reduced the distortion derived from inversion operation, a parallel Melfrequency and logmagnitude (PMLM) modeling. Estimated clean speech waveform from a noisy signal, an inversion from the Melfrequency domain to the spectral domain is required which introduces distortion artifacts in the spectrum estimation and the filtering.
Input audio is considered and the enhancement is played.
Following figure shows log magnitude plot of the original speech.
Fig 4: Log magnitude plot of the original speech
Following are the spectrograms of noisy and filtered speech.
Fig5: Spectrogram of noisy and filtered speech
Fig6: Spectrogram of signal after Weiner filter CONCLUSION
In this paper, we present a HMMbased speech enhancement using VTS and PMLM modeling in Mel frequency domain. The VTS is used to estimate the mean vectors and covariance matrices of noisy speech HMM more accurately, and the PMLM modeling is used to reduce the spectral distortion resulted by the inversion from Mel frequency domain to spectral domain. As obviously shown in the evaluation results, the performance of proposed method is better than the reference methods.
Demonstrating that the proposed methods reduce the annoying nonstationary noise associated with the enhanced speech.
Generating the spectrograms and calculating the mean and co variance vectors.
ACKNOWLEDGMENT
We are really thankful to our guide without which the accomplishment of the task would have never been possible. we thankful to all other helpful people for providing us relevant information and necessary clarifications.
REFERENCES

Y. Ephraim, D. Melah and B. Juang, On the Application of Hidden Markov Models for Enhancing Noisy Speech, Acoustics, Speech and Signal Processing, IEEE Transactions on, 1989, vol. 37, pp.18461856.

Y. Ephraim, A minimum mean square error approach for speech enhancement, Acoustics, Speech and Signal Processing (ICASSP), 1990 International Conference on. IEEE, 1990, pp.829832

H. Sameti, H. Sheikhzadeh, L. Deng and L. B. Robert, HMMbased strategies for enhancement of speech signals embedded in nonstationary noise, Speech and Audio Processing, IEEE Transactions on, 1998, vol. 6, pp.445455.

M. Nilsson, M. Dahl and I. Claesson, HMMbased speech enhancement applied in nonstationary noise using cepstral features and lognormal approximation, Proc. DSPCS, 2003, pp.8286.

H. Veisi and H. Sameti, A Parallel Cepstral and Spectral Modeling for HMMbased Speech Enhancement, Digital Signal Processing (DSP), 2011 17th International Conference on. IEEE, 2011, pp.16.

Zhenzhen Gao, Changchun Bao, Feng Bao, Maoshen, HMM based speech enhancement using vector Taylor series and parallel modeling in Mel frequency domain, IEEE ,2014

Berdugo, B. and Cohen, I., Noise estimation by minima controlled recursive averaging for robust speech enhancement, IEEE Signal Proc. Letters, vol. 9, no. 1, pp. 1215, Jan. 2002.

R. Martin, Speech enhancement using a minimum meansquare error shorttime spectral amplitude estimation, in Proc. IEEE Int. Conf. Acoust Speech, Signal Processing, pp. 504512, 2002.

Cohen, Noise spectrum estimation in adverse environments Improved minia controlled recursive averaging, IEEE Trans. on speech and audio processing, vol. 11, no. 5, pp. 466475, Sept. 2003.

Barinder pal singh, Dr shahi Bhushan, Mr. Karan Mahajan,speech enhancement using Hidden Markov model, IJCSMC, Vol. 4, Issue. 7, July 2015.

Hadi veisi,Hossein sameti, speech enhancement using hidden Markov models in Melfrequency domain, speech communication,vol.55,Issue 2,feb 2013.

Daniel Dzibela, Armin Sehr Hidden Markov Model Based Speech Enhancement, 2017 IEEE.

Akhirao Kato and Ben Milner Using Hidden Markov Models for Speech Enhancement 2014 ICSA

Przybyla Dymarski Hidden Markov Models, Theory and Applications 2011.