HI nsh,

>At least now we have estimation of the model accuracy, on the 1/10 test

>set with a custom trigram lm trained on the test prompts it has the

>following quality:

> TOTAL Words: 28112 Correct: 25767 Errors: 3158
>TOTAL Percent correct = 91.66% Error = 11.23% Accuracy = 88.77%
>TOTAL Insertions: 813 Deletions: 415 Substitutions: 1930

> Not bad, but I suppose we can raise the accuracy to 97% if we'll try to

>optimize training.

Do these numbers include the problem prompts too, or did you omit them?  i.e. is all we have to do to get to 97% is remove or correct the offending submission prompts?

thanks,

Ken

--- (Edited on 9/9/2008 12:13 pm [GMT-0400] by kmaclean) ---

> Do these numbers include the problem prompts too, or did you omit them?  i.e. is all we have to do to get to 97% is remove or correct the offending submission prompts?

Surely not, we have to correct transcriptions, optimize training paramters, train MLLT and LDA transformations.

Btw, David recently tested voxforge-en model on wsj test set, so here is the real result:

 test 20k, Sphinx3.7, bigrams:

==> voxforge_s3_test20k/voxforge_s3_test20k.align <==
TOTAL Words: 5645 Correct: 4557 Errors: 1230
TOTAL Percent correct = 80.73% Error = 21.79% Accuracy = 78.21%
TOTAL Insertions: 142 Deletions: 159 Substitutions: 929

test 5k, Sphinx3.7, trigrams:

==> voxforge_s3_test5k/voxforge_s3_test5k.align <==
TOTAL Words: 5354 Correct: 4880 Errors: 562
TOTAL Percent correct = 91.15% Error = 10.50% Accuracy = 89.50%
TOTAL Insertions: 88 Deletions: 65 Substitutions: 409

 Original WSJ results:

 SI-84 (14 hours), 2800 senones, 8 Gaussians, trigrams, Sphinx3, test 5k:

==> si84_sphinx3/si84_sphinx3.align <==
TOTAL Words: 5354 Correct: 5079 Errors: 325
TOTAL Percent correct = 94.86% Error = 6.07% Accuracy = 93.93%
TOTAL Insertions: 50 Deletions: 56 Substitutions: 219

SI-284 (80 hours?), 3000 senones, 32 Gaussians, bigrams, Sphinx3, test20k:

==> si284_20k_sphinx3/si284_20k_sphinx3.align <==
TOTAL Words: 5645 Correct: 5164 Errors: 559
TOTAL Percent correct = 91.48% Error = 9.90% Accuracy = 90.10%
TOTAL Insertions: 78 Deletions: 68 Substitutions: 413

--- (Edited on 9/10/2008 4:39 pm [GMT-0500] by nsh) ---

I managed to use Opera instead of Mozilla to write this one.

Ken, can you please update the model:

http://www.mediafire.com/download.php?atmdlrdt0om

MLLT training made this one better:

TOTAL Words: 28420 Correct: 26989 Errors: 1929
TOTAL Percent correct = 94.96% Error = 6.79% Accuracy = 93.21%
TOTAL Insertions: 498 Deletions: 362 Substitutions: 1069

--- (Edited on 9/22/2008 12:14 pm [GMT-0500] by nsh) ---