Speech Recognition Engines

Flat
Speaker Adaptation in HTK 3.4.1
User: Eric
Date: 6/24/2015 4:47 pm
Views: 7518
Rating: 1

Hi All,

 

I'm attempting to adapt my tied-state, context-dependent triphone system to a set of data from a novel speaker via mllr/cmll. Models have 16 mixture components, except for sil which is 32. I'm running 3.4.1 in an ubuntu environment. I'm following the guidelines given in the HTK book 3.4.1 pages 43-45.

I have no problem creating the regression tree or creating the global transform. When using the global transform as an input transform for the second pass of HERest to create either a new transform or an updated model set, I encounter a segmentation fault. Find below the results of the calls to HHEd and HERest

-------------------------------------------------------


HHEd -A -T 1 -V -D -H model/hmm13/globals -H model/hmm13/hmmdefs -M adapt adapt/regtree.hed tiedlist

No HTK Configuration Parameters Set


HTK Version Information
Module     Version    Who    Date      : CVS Info
HHEd       3.4.1      CUED   12/03/09  : $Id: HHEd.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $
HShell     3.4.1      CUED   12/03/09  : $Id: HShell.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HMem       3.4.1      CUED   12/03/09  : $Id: HMem.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HLabel     3.4.1      CUED   12/03/09  : $Id: HLabel.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HMath      3.4.1      CUED   12/03/09  : $Id: HMath.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HSigP      3.4.1      CUED   12/03/09  : $Id: HSigP.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HWave      3.4.1      CUED   12/03/09  : $Id: HWave.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HAudio     3.4.1      CUED   12/03/09  : $Id: HAudio.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HVQ        3.4.1      CUED   12/03/09  : $Id: HVQ.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HModel     3.4.1      CUED   12/03/09  : $Id: HModel.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $
HParm      3.4.1      CUED   12/03/09  : $Id: HParm.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HUtil      3.4.1      CUED   12/03/09  : $Id: HUtil.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $

HHEd
 37443/17518 Models Loaded [5 states max, 32 mixes max]

LS stats
 Loading state occupation stats
  Stats loaded for 10211 models
  Mean Occupation Count = 362.313977

RC 32 rtree
 Building regression tree with 32 terminals
Creating regression class tree with ident rtree.tree and baseclass rtree.base
Splitting Node 1, score 9.104666e+07
Splitting Node 3, score 4.685049e+07
Splitting Node 2, score 3.131549e+07
Splitting Node 4, score 2.732998e+07
Splitting Node 7, score 1.813764e+07
Splitting Node 5, score 1.555596e+07
Splitting Node 8, score 1.510295e+07
Splitting Node 9, score 1.021550e+07
Splitting Node 6, score 1.014084e+07
Splitting Node 11, score 8.686515e+06
Splitting Node 15, score 7.892073e+06
Splitting Node 10, score 7.627271e+06
Splitting Node 12, score 7.467690e+06
Splitting Node 13, score 6.619383e+06
Splitting Node 14, score 6.289132e+06
Splitting Node 20, score 6.007492e+06
Splitting Node 18, score 5.535949e+06
Splitting Node 17, score 5.024976e+06
Splitting Node 16, score 4.456918e+06
Splitting Node 30, score 4.197672e+06
Splitting Node 27, score 4.112460e+06
Splitting Node 22, score 4.030436e+06
Splitting Node 19, score 3.639844e+06
Splitting Node 35, score 3.617287e+06
Splitting Node 28, score 3.460538e+06
Splitting Node 25, score 3.456282e+06
Splitting Node 24, score 3.371160e+06
Splitting Node 23, score 3.103943e+06
Splitting Node 32, score 2.870081e+06
Splitting Node 26, score 2.683608e+06
Splitting Node 29, score 2.571511e+06
Edit Complete

No HTK Configuration Parameters Set

HERest -A -T 1 -V -D -C model/config -C adapt/config.global -S hvite_test/test_mfc.scp -I hvite_test/cmc_wintri.mlf -H model/hmm13/globals -H model/hmm13/hmmdefs -K adapt/xforms cmllr1 -J adapt -u a -h */%%%_*.mfc tiedlist

HTK Configuration Parameters[18]
  Module/Tool     Parameter                  Value
# HMODEL          TRACE                        512
# HADAPT          TRACE                         61
# HADAPT          KEEPXFORMDISTINCT              TRUE
# HADAPT          ADAPTKIND                   BASE
# HADAPT          BASECLASS                   glob
# HADAPT          USEBIAS                     TRUE
# HADAPT          TRANSKIND               MLLRMEAN
#                 ENORMALISE                  TRUE
#                 NUMCEPS                       12
#                 CEPLIFTER                     22
#                 NUMCHANS                      26
#                 PREEMCOEF               0.970000
#                 USEHAMMING                  TRUE
#                 WINDOWSIZE         250000.000000
#                 SAVEWITHCRC                 TRUE
#                 SAVECOMPRESSED              TRUE
#                 TARGETRATE         100000.000000
#                 TARGETKIND            MFCC_E_D_A


HTK Version Information
Module     Version    Who    Date      : CVS Info
HERest     3.4.1      CUED   12/03/09  : $Id: HERest.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $
HShell     3.4.1      CUED   12/03/09  : $Id: HShell.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HMem       3.4.1      CUED   12/03/09  : $Id: HMem.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HMath      3.4.1      CUED   12/03/09  : $Id: HMath.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HSigP      3.4.1      CUED   12/03/09  : $Id: HSigP.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HAudio     3.4.1      CUED   12/03/09  : $Id: HAudio.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HWave      3.4.1      CUED   12/03/09  : $Id: HWave.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HVQ        3.4.1      CUED   12/03/09  : $Id: HVQ.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HLabel     3.4.1      CUED   12/03/09  : $Id: HLabel.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HModel     3.4.1      CUED   12/03/09  : $Id: HModel.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $
HParm      3.4.1      CUED   12/03/09  : $Id: HParm.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HTrain     3.4.1      CUED   12/03/09  : $Id: HTrain.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HUtil      3.4.1      CUED   12/03/09  : $Id: HUtil.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HFB        3.4.1      CUED   12/03/09  : $Id: HFB.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HAdapt     3.4.1      CUED   12/03/09  : $Id: HAdapt.c,v 1.2 2006/12/07 11:09:07 mjfg Exp $
HMap       3.4.1      CUED   12/03/09  : $Id: HMap.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $

HERest  ML Updating: XForms

 System is SHARED
37443 Logical/17518 Physical Models Loaded, VecSize=39
2 MMF input files
  Using baseclass macro "glob" from file adapt/glob
Attached 70672 XFormInfo structures
Attached 70669 RegAcc structures
Pruning-Off
 Processing Data: cmc_1.mfc; Label cmc_1.lab
 Utterance prob per frame = -6.767935e+01
 Processing Data: cmc_2.mfc; Label cmc_2.lab
 Utterance prob per frame = -6.676580e+01
 Processing Data: cmc_3.mfc; Label cmc_3.lab
 Utterance prob per frame = -6.482462e+01
 Processing Data: cmc_4.mfc; Label cmc_4.lab
 Utterance prob per frame = -6.825621e+01
 Processing Data: cmc_5.mfc; Label cmc_5.lab
 Utterance prob per frame = -6.721992e+01
 Processing Data: cmc_6.mfc; Label cmc_6.lab
 Utterance prob per frame = -6.683469e+01
 Processing Data: cmc_7.mfc; Label cmc_7.lab
 Utterance prob per frame = -6.876072e+01
 Processing Data: cmc_8.mfc; Label cmc_8.lab
 Utterance prob per frame = -6.697883e+01
 Processing Data: cmc_9.mfc; Label cmc_9.lab
 Utterance prob per frame = -6.724328e+01
 Processing Data: cmc_10.mfc; Label cmc_10.lab
 Utterance prob per frame = -6.706760e+01
 Processing Data: cmc_11.mfc; Label cmc_11.lab
 Utterance prob per frame = -6.580224e+01
 Processing Data: cmc_12.mfc; Label cmc_12.lab
 Utterance prob per frame = -6.688803e+01
 Processing Data: cmc_13.mfc; Label cmc_13.lab
 Utterance prob per frame = -6.504648e+01
Generating transform cmc (1)
  Using baseclass macro "glob" from file adapt/glob
Estimated XForm 1 using 40720.329491 observations
Reestimation complete - average log prob per frame = -6.693842e+01 (40748 frames)

HTK Configuration Parameters[18]
  Module/Tool     Parameter                  Value
  HMODEL          TRACE                        512
  HADAPT          TRACE                         61
  HADAPT          KEEPXFORMDISTINCT              TRUE
  HADAPT          ADAPTKIND                   BASE
  HADAPT          BASECLASS                   glob
  HADAPT          USEBIAS                     TRUE
  HADAPT          TRANSKIND               MLLRMEAN
                  ENORMALISE                  TRUE
                  NUMCEPS                       12
                  CEPLIFTER                     22
                  NUMCHANS                      26
                  PREEMCOEF               0.970000
                  USEHAMMING                  TRUE
                  WINDOWSIZE         250000.000000
                  SAVEWITHCRC                 TRUE
                  SAVECOMPRESSED              TRUE
                  TARGETRATE         100000.000000
                  TARGETKIND            MFCC_E_D_A

HERest -A -T 1 -D -V -a -C model/config -C adapt/config.rc -S hvite_test/test_mfc.scp -I hvite_test/cmc_wintri.mlf -H model/hmm13/globals -H model/hmm13/hmmdefs -J adapt/xforms cmllr1 -K adapt/xforms cmllr2 -J adapt -u a -h */%%%_*.mfc tiedlist

HTK Configuration Parameters[19]
  Module/Tool     Parameter                  Value
# HMODEL          TRACE                        512
# HADAPT          TRACE                         61
# HADAPT          KEEPXFORMDISTINCT              TRUE
# HADAPT          SPLITTHRESH          1000.000000
# HADAPT          ADAPTKIND                   TREE
# HADAPT          REGTREE               rtree.tree
# HADAPT          USEBIAS                     TRUE
# HADAPT          TRANSKIND               MLLRMEAN
#                 ENORMALISE                  TRUE
#                 NUMCEPS                       12
#                 CEPLIFTER                     22
#                 NUMCHANS                      26
#                 PREEMCOEF               0.970000
#                 USEHAMMING                  TRUE
#                 WINDOWSIZE         250000.000000
#                 SAVEWITHCRC                 TRUE
#                 SAVECOMPRESSED              TRUE
#                 TARGETRATE         100000.000000
#                 TARGETKIND            MFCC_E_D_A


HTK Version Information
Module     Version    Who    Date      : CVS Info
HERest     3.4.1      CUED   12/03/09  : $Id: HERest.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $
HShell     3.4.1      CUED   12/03/09  : $Id: HShell.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HMem       3.4.1      CUED   12/03/09  : $Id: HMem.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HMath      3.4.1      CUED   12/03/09  : $Id: HMath.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HSigP      3.4.1      CUED   12/03/09  : $Id: HSigP.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HAudio     3.4.1      CUED   12/03/09  : $Id: HAudio.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HWave      3.4.1      CUED   12/03/09  : $Id: HWave.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HVQ        3.4.1      CUED   12/03/09  : $Id: HVQ.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HLabel     3.4.1      CUED   12/03/09  : $Id: HLabel.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HModel     3.4.1      CUED   12/03/09  : $Id: HModel.c,v 1.2 2006/12/07 11:09:08 mjfg Exp $
HParm      3.4.1      CUED   12/03/09  : $Id: HParm.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HTrain     3.4.1      CUED   12/03/09  : $Id: HTrain.c,v 1.1.1.1 2006/10/11 09:54:58 jal58 Exp $
HUtil      3.4.1      CUED   12/03/09  : $Id: HUtil.c,v 1.1.1.1 2006/10/11 09:54:59 jal58 Exp $
HFB        3.4.1      CUED   12/03/09  : $Id: HFB.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $
HAdapt     3.4.1      CUED   12/03/09  : $Id: HAdapt.c,v 1.2 2006/12/07 11:09:07 mjfg Exp $
HMap       3.4.1      CUED   12/03/09  : $Id: HMap.c,v 1.1.1.1 2006/10/11 09:54:57 jal58 Exp $

HERest  ML Updating: XForms

 System is SHARED
37443 Logical/17518 Physical Models Loaded, VecSize=39
2 MMF input files
  Using baseclass macro "rtree.base" from file adapt/rtree.base
  Using regtree macro "rtree.tree" from file adapt/rtree.tree
Attached 70672 XFormInfo structures
Attached 70669 RegAcc structures
Pruning-Off
Using input transform cmc
  Using baseclass macro "glob" from file adapt/glob
  Using xform macro "cmc.cmllr1" from file adapt/xforms/cmc.cmllr1
Attached 70669 AInfo structures
Created 0 ObsCaches (of 1 possible)
Attached 70672 MInfo structure

Segmentation fault (core dumped)


-------------------------------------------------------

At which point I encounter the segmentation fault...

 

Do you have any suggestions as to why I might be encountering this problem? Interestingly, I have succeeded with an identical process on a different language model with different phones on this same machine, so I don't believe it's an issue with the installation of HTK. I think the suspect may be the regression tree, though I'm not sure. Additionally, I've found someone else with what appears to be the same problem here.

 

Thanks for your time!

Eric

--- (Edited on 6/24/2015 4:48 pm [GMT-0500] by Visitor) ---

Re: Speaker Adaptation in HTK 3.4.1
User: colbec
Date: 6/26/2015 8:48 am
Views: 461
Rating: 1

Have you tried to get any information from your core dump with gdb? That may give you some information regarding where (HERest?) failed. Your observation regarding phones may be pertinent, since it involves strings which may be badly interpreted by pointers.

--- (Edited on 2015-06-26 9:48 am [GMT-0400] by colbec) ---

Re: Speaker Adaptation in HTK 3.4.1
User: eric
Date: 7/1/2015 2:07 pm
Views: 124
Rating: 1

Sorry for the late reply Colbec.


Running the command from within gdb:

##############

gdb /usr/local/bin/HEREst

(gdb)  run -A -T 1 -D -a -C model/config -C adapt/config.rc -S scripts/mfc.scp -I data/wintri.mlf -H model/hmm13/globals -H model/hmm13/hmmdefs -M adapt -J adapt/xforms cmllr1 -J adapt -h '*/%%%%%%*_*.mfc' tiedlist

##############

Yields the output:

Program received signal SIGSEGV, Segmentation fault.
CompXForm (mp=0x7ffff78f2d20, xform=0x7ffff5a59148, ai=0x0) at HAdapt.c:2299
2299      if (ai->next != NULL) { /* There's a parent transform */

##############


calling: (gdb) where

yields:

##############

#0  CompXForm (mp=0x7ffff78f2d20, xform=0x7ffff5a59148, ai=0x0) at HAdapt.c:2299
#1  0x000000000040dff0 in ApplyHMMSetXForm (hset=<value optimized out>, xform=0x7ffff5a59148) at HAdapt.c:4112
#2  0x0000000000410a1a in UpdateSpkrStats (hset=0x7fffffffdc10, xfinfo=0x684160, datafn=0x8784a0 "./data/ral126_BR_14929.mfc") at HAdapt.c:4501
#3  0x0000000000405143 in main (argc=<value optimized out>, argv=<value optimized out>) at HERest.c:524

##############

I'm inexperienced with reading gdb output, is there anything obvious that I'm missing?

I'm fairly sure now that the fault is due to memory usage in HAdapt, but whether or not that memory usage is due to a bug or to my data is unclear. Unfortunately, the segmentation fault happens too quickly for me to monitor the memory usage via top.

Rolling back to monophones did allow the adaptation to complete successfully.

--- (Edited on 7/1/2015 2:07 pm [GMT-0500] by eric) ---

--- (Edited on 7/1/2015 2:07 pm [GMT-0500] by eric) ---

Re: Speaker Adaptation in HTK 3.4.1
User: colbec
Date: 7/2/2015 3:58 pm
Views: 1942
Rating: 1

It seems to be pointing at HAdapt, on or about line 2299.

A quick google for htk hadapt 2299 does not show anything much to me, so if it is a bug it must be a very obscure one. I guess you could ask yourself what you may be doing that is way off in left field.

As we guessed, strings and pointers seem to be involved. The var ai seems to be a struct that holds a list (LIFO FIFO queue or something) that at some points to a memory location in ai->next and if next is not in the list then clunk.

Two suggestions: put the question (now that you have details) to the folks at HTK (see for example http://htk.eng.cam.ac.uk/develop/report_bug.shtml) and continue what you are doing, changing the data inputs to see if patterns emerge.

--- (Edited on 2015-07-02 4:58 pm [GMT-0400] by colbec) ---

Re: Speaker Adaptation in HTK 3.4.1
User: maituan2012
Date: 3/21/2017 3:59 am
Views: 185
Rating: 0

hello, I'm now have the same problem like you did. Did you have any solution ?

--- (Edited on 3/21/2017 3:59 am [GMT-0500] by maituan2012) ---

Re: Speaker Adaptation in HTK 3.4.1
User: maituan2012
Date: 3/31/2017 3:19 am
Views: 1834
Rating: 0

hello,


did you find any method to remove this problem, i really need it.

--- (Edited on 3/31/2017 3:19 am [GMT-0500] by maituan2012) ---

PreviousNext