Hi,

I'm sure there are plans for creating testing corpora of various difficulties for VoxForge. Here is my suggestion for one:

Spoken numbers, let's say between one and one million

Reasons:

• Easy task, you can expect over 95% accuracy
• Can be recognized with a grammar => no need for language model tweaking and no need for special recognizers, HVite is sufficient

Creating a grammar for english numbers should be straightforward, here is a place to start:

http://www.perlmonks.org/?node_id=506028

When generating the prompts, do not use HSGen since this leads to weird numbers like "one hundred thousand and three", use a random number generator and then convert the numbers to words. A convertor can be found e.g. here:

http://search.cpan.org/~sburke/Lingua-EN-Numbers-1.01/lib/Lingua/EN/Numbers.pm

Also native and non native speakers should be clearly separated so that you can see the difference in accuracy.

--- (Edited on 3/11/2009 3:57 am [GMT-0500] by tpavelka) ---

--- (Edited on 3/11/2009 4:06 am [GMT-0500] by tpavelka) ---

Hi tpavelka,

>I'm sure there are plans for creating testing corpora of various difficulties

>for VoxForge.

Well, actually, my current focus is facilitating the collection of speech and setting-up a way to categorize submissions so that novel sub-corpora can be generated from it (noisy, clean, non-native ...). 

If anyone wants to take on corpora testing, please let me know.

>Spoken numbers, let's say between one and one million

thanks, added to Trac Ticket #16 - Corpora testing

Ken

--- (Edited on 3/13/2009 9:50 pm [GMT-0400] by kmaclean) ---

Hi Ken,

> If anyone wants to take on corpora testing, please let me know.

I just got a grammar for English numbers from a coleague. I also have a parser to check whether a sentence can be parsed by that grammar. I can write the prompt generator and send it to you so you can either use it in your code or use the generated prompts.

Having the grammar the testing should be pretty straightforward (given that you also have the recordings).

Tomas

--- (Edited on 3/16/2009 6:48 am [GMT-0500] by tpavelka) ---

HI tpavelka,

>I can write the prompt generator and send it to you so you can either use it

>in your code or use the generated prompts.

>Having the grammar the testing should be pretty straightforward (given that

>you also have the recordings).

Forgive my ignorance here, but do you mean that you have a script that can generate numeric prompts, which can then be included in the Speech Submission applet, so we can get people to record themselves reading these prompts, and use these for testing?  I am not sure I understand how just having a prompt generator also gives me access to the speech audio recording...

thanks,

Ken

--- (Edited on 3/17/2009 10:36 am [GMT-0400] by kmaclean) ---

> Forgive my ignorance here, but do you mean that you have a

> script that can generate numeric prompts, which can then be

> included in the Speech Submission applet, so we can get people

> to record themselves reading these prompts, and use these for testing?

Sorry, I did not write it very clearly. This is what I meant. The generator only makes text prompts which you have to record later.

Before I send you the generator I want to make sure that I have a grammar that can parse these prompts. For that I need a parser, which was written by one of my coleagues, but I found a few bugs fhich he is fixing right now. After it is corrected I can use it to check whether the grammar covers the generated prompts.

The you can use the prompts in the speech submission applet.

When you have a large enough corpus all you need to do is convert the grammar into HTK word net and run HVite with it.

Tomas

--- (Edited on 3/17/2009 10:19 am [GMT-0500] by tpavelka) ---

Ok,

finally the convertor is here. Writing the generator was pretty strainght forward using Lingua:EN:Numbers. I got the grammar from a coleague who used it in a different project. It is in JSGF format so I wrote a simple convertor into the EBNF format used by HTK.

The tricky part was ensuring that the grammar covers all the generated sentences. For that I used a parser that can convert the sentences back into numbers, after that the number can be compared to the original one.

Tomas 

--- (Edited on 3/18/2009 9:47 am [GMT-0500] by tpavelka) ---

Hi Tomas,

Sorry for the delay in getting back to you, been travelling...

>finally the convertor is here. Writing the generator was pretty strainght

>forward using Lingua:EN:Numbers.

Thanks a million for this!

I cannot add these new prompts to the Java applet just yet, but will do so after the conversion of the VoxForge website to Drupal.   Added  Ticket #454 to track this.

thanks again,

Ken

--- (Edited on 3/30/2009 2:09 pm [GMT-0400] by kmaclean) ---