[Home]

Summary:ASTERISK-13674: [patch] Generic Speech API Engine - Sphinx
Reporter:Christopher Jansen (scribblej)Labels:
Date Opened:2009-03-01 20:54:38.000-0600Date Closed:2011-06-07 14:03:18
Priority:MajorRegression?No
Status:Closed/CompleteComponents:Resources/res_speech/NewFeature
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) server.tgz
( 1) sphinx.patch
Description:Generic Speech Engine API plugin for using the CMU Sphinx speech recognizer as a speech rec system in Asterisk.  

This consists of two parts; the engine which plugs into Asterisk works as a client to a Sphinx server which can be run on another machine, so there is no resource contention.  Additionally having a split architecture allows Digium to decide whether the server portion can or should be included in Asterisk as a separate decision from the engine plugin.



****** ADDITIONAL INFORMATION ******

At this point my questions are many and my code is likely awful.  Primarily, I'm not sure what to do with the server portion, how to submit it and where it should go in the long-run.  In addition, the engine plugin is my first foray into both use of and programming for Asterisk, so it's likely I got some fundamental things wrong which I cannot identify myself.

Especially, I make use of some socket code here, using bsd-style non-blocking tcp socket code, which probably needs to be done in a more portable fashion?
Comments:By: Christopher Jansen (scribblej) 2009-03-01 20:57:43.000-0600

I'm not sure the proper way to denote this, but there is an equivalent patch against 1.4.21 and more documentation and explanation on my website, at http://scribblej.com/svn/ (which is not an svn repo, just a poorly-chosen directory name)
By: Leif Madsen (lmadsen) 2009-05-20 09:20:01

Would it be possible for you to write up some simple documentation so that it is easier for someone to test this? What exactly are we getting out of this patch? Does it allow us the ability to use Sphinx as a text-to-speech system a lot easier from the dialplan?

I just stumbled across this while doing bug triage for issues that haven't been moved forward in quite some time, and would be willing to do some testing and such here if you could elaborate a bit. Thanks in advance!
By: Ronald Chan (loloski) 2009-06-10 03:28:43

this was a great addition to asterisk arsenal, if this can be included in the main stream.

I get my basic speech recognition needs on this http://www.syednetworks.com/asterisk-integration-with-sphinx-voice-recognition-system

It's working as advertise, but not a clean solution since it was based on the combiation of AGI script.

I will attach a compilation failure against this patch.


SVN-trunk-r199696

the patch tested was download from http://scribblej.com/svn/ website.
By: Ronald Chan (loloski) 2009-06-10 03:51:56

whoops, i was able to alleviate the situation by try to recompile this patch. the attach patch was successfully patched and compiled, now time to test.....
By: Ronald Chan (loloski) 2009-06-10 03:58:16

missing sphinx.conf segfault asterisk ... more to come later...
By: Ronald Chan (loloski) 2009-06-11 00:13:44

scribblej,

I'm having a problem on starting astsphinx server, please check relevant logs for details. thanks


INFO: acmod.c(232): Parsed model-specific feature parameters from /opt/sphinx/communicator/feat.params
INFO: fe_interface.c(288): You are using the internal mechanism to generate the seed.
INFO: feat.c(848): Initializing feature stream to type: 's2_4x', ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0
INFO: mdef.c(520): Reading model definition: /opt/sphinx/communicator/mdef
INFO: s2_semi_mgau.c(664): Loading senones from dump file /opt/sphinx/communicator/sendump
INFO: s2_semi_mgau.c(680): BEGIN FILE FORMAT DESCRIPTION
INFO: s2_semi_mgau.c(709): Rows: 256, Columns: 6256
INFO: s2_semi_mgau.c(717): Using memory-mapped I/O for senones
INFO: s2_semi_mgau.c(1121): Maximum top-N: 4 Top-N beams: 0 0 0 0
INFO: dict.c(232): Allocating 20 placeholders for new OOVs
ERROR: "dict.c", line 553: 'five': Unknown phone 'AY1'
ERROR: "dict.c", line 437: Failed to add five to dictionary
ERROR: "dict.c", line 553: 'four': Unknown phone 'AO1'
ERROR: "dict.c", line 437: Failed to add four to dictionary
ERROR: "dict.c", line 553: 'one': Unknown phone 'AH1'
ERROR: "dict.c", line 437: Failed to add one to dictionary
ERROR: "dict.c", line 553: 'one(1)': Unknown phone 'AH1'
ERROR: "dict.c", line 437: Failed to add one(1) to dictionary
ERROR: "dict.c", line 553: 'three': Unknown phone 'IY1'
ERROR: "dict.c", line 437: Failed to add three to dictionary
ERROR: "dict.c", line 553: 'two': Unknown phone 'UW1'
ERROR: "dict.c", line 437: Failed to add two to dictionary
INFO: dict.c(491):      0 = words in file [/opt/sphinx/dictionary/dictionary]
Restart checking timeout (1244698982 - 1244698979 > 2), 15
DIFF: 1.500000, LIMIT: 7.500000, RESTARTS: 10.000000
Spawning too quickly
Killed
By: Ronald Chan (loloski) 2009-06-11 06:00:39

Name/username              Host            Dyn Nat ACL Port     Status
100/100                    124.106.206.96   D   N      5060     OK (36 ms)
101/101                    (Unspecified)    D   N      5060     UNKNOWN
2 sip peers [Monitored: 1 online, 1 offline Unmonitored: 0 online, 0 offline]
   -- Executing [4000@internal:1] Answer("SIP/100-b7622570", "") in new stack
   -- Executing [4000@internal:2] SpeechCreate("SIP/100-b7622570", "Sphinx") in new stack
   -- Executing [4000@internal:3] SpeechActivateGrammar("SIP/100-b7622570", "4828") in new stack
   -- Executing [4000@internal:4] SpeechStart("SIP/100-b7622570", "") in new stack
   -- Executing [4000@internal:5] SpeechBackground("SIP/100-b7622570", "vm-goodbye,10") in new stack
[Jun 11 19:29:51] NOTICE[1797]: res_speech_sphinx.c:307 sphinx_sread: Score: 825 Result: 'YES'
[Jun 11 19:29:51] NOTICE[1797]: res_speech_sphinx.c:310 sphinx_sread: New result with lower score; ignoring.
[Jun 11 19:29:51] NOTICE[1797]: res_speech_sphinx.c:307 sphinx_sread: Score: 826 Result: 'YES'
[Jun 11 19:29:51] NOTICE[1797]: res_speech_sphinx.c:307 sphinx_sread: Score: 833 Result: 'YES'
[Jun 11 19:29:51] NOTICE[1797]: res_speech_sphinx.c:307 sphinx_sread: Score: 856 Result: 'YES'
[Jun 11 19:29:51] NOTICE[1797]: res_speech_sphinx.c:307 sphinx_sread: Score: 880 Result: 'YES'
   -- Executing [4000@internal:6] SpeechDeactivateGrammar("SIP/100-b7622570", "4828") in new stack
   -- Executing [4000@internal:7] Verbose("SIP/100-b7622570", "YES") in new stack
YES
   -- Executing [4000@internal:8] Hangup("SIP/100-b7622570", "") in new stack
 == Spawn extension (internal, 4000, 8) exited non-zero on 'SIP/100-b7622570'
By: Ronald Chan (loloski) 2009-06-11 06:02:11

after a long day i finally test it and it works, thanks to bharani on #cmusphinx @ irc.freenode.net
By: Ronald Chan (loloski) 2009-06-14 08:08:50

can someone comment on this please?, where we go from here?
By: Leif Madsen (lmadsen) 2009-06-16 13:30:58

loloski: can you provide some documentation on how you got this to work so that we don't have other people running into similar issues as what you ran into? Once that is done, I can test it and see if the documentation makes sense and works, then I can merge it to the patch.

Then we can mark this as ready for testing, and eventually ready for review.

Thanks!
By: Ronald Chan (loloski) 2009-06-21 08:03:57

leif,

I will try my best to make a small documentation on this, just give me a time because english is not my native tongue. I'll try within the week or so.
By: Leif Madsen (lmadsen) 2009-06-22 09:34:54

loloski: no worries! Just some simple notes will be fine to give me the instructions. I will update the documentation to correct any language issues, so don't worry about those.

Thanks!
By: Leif Madsen (lmadsen) 2009-07-15 10:18:42

Just wanted to ping this issue again as I wouldn't mind testing this out and determining how we're going to move this forward. Thanks!
By: Sebastian Gutierrez (sum) 2009-08-12 18:00:12

I'm also interested on testing this! I'm using 1.6.2
By: Leif Madsen (lmadsen) 2009-11-04 11:47:11.000-0600

loloski: any change you're going to be able to follow up on some documentation here? Otherwise I think I'm going to suspend this issue for now.
By: Ronald Chan (loloski) 2009-11-04 23:10:13.000-0600

lmadsen,

I'm really sorry for extremely delay on this, you may suspend this issue for now, since i was in an odd situation that I can make my promise with you. anyway this stuff is working for a simple case but not on a real world application.
By: Leif Madsen (lmadsen) 2009-11-05 09:33:43.000-0600

It is with much sadness that I am closing this issue, as it looked so promising! If someone is able to pick this up and continue with it to the end, it would be much appreciate! Just find a bug marshal on #asterisk-bugs on the Freenode IRC server at irc.freenode.net to have this issue reopened.

Thanks!