[Home]

Summary:ASTERISK-12981: [patch] No way to tune talker optimization in meetme, causes users to get cut off while they're still talking
Reporter:Dave Miller (justdave)Labels:
Date Opened:2008-10-29 13:47:40Date Closed:2009-05-14 12:07:57
Priority:MajorRegression?No
Status:Closed/CompleteComponents:Applications/app_meetme
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 13801-revisit.diff
( 1) 20081119__bug13801.diff.txt
( 2) meetme-optimize-talker-off-trunk.txt
Description:I enabled 'o' talker optimization on my conference rooms because the documentation in 1.4 says the feature will be permanently enabled in Asterisk 1.6 with no way to disable it, so I figured we should probably get used to it. However, if it works like this we'll have to never upgrade to 1.6.  We get constant complaints about people getting cut off while still talking in the conferences, and I can't find any way to tune what it considers "talking". If the feature is going to be permanently enabled, we at least need some way to tune how sensitive it is.
Comments:By: Russell Bryant (russell) 2008-10-29 15:24:14

This is the first time I've heard of problems with it.  If that is the case, we should probably change Asterisk 1.6 to make it optional again.
By: mdu113 (mdu113) 2008-10-29 15:44:40

I can confirm the problem. It's actually so bad that after a few experiments we have completely given up on using that optimization feature. It may be ok for providing free/cheap conferences, but not for the cases when you need good voice quality.
I vote for making it optional.
By: Tilghman Lesher (tilghman) 2008-11-19 01:35:55.000-0600

justdave:  Please try this patch for 1.4.  It makes the silence threshold settable to a different value.  Please try lower values and see if that makes the silence detection work better for you.  In 1.6, this will be settable via /etc/asterisk/dsp.conf, although we may need to ensure that it's obeyed in meetme.
By: mdu113 (mdu113) 2008-11-19 09:48:21.000-0600

Will this setting allow setting threshold to 0/infinity effectively disabling optimization?
By: Tilghman Lesher (tilghman) 2008-11-19 10:43:41.000-0600

mdu113:  actually, I suppose the values would be 0 to disable, 2147483647 to make it all but impossible to detect anything but silence.
By: dea (dea) 2008-11-19 21:53:03.000-0600

I am seeing the same thing in 1.6.0.1

I thought it was unrelated to talker optimization, since early tests
had been positive, and I went looking for other causes.

The patch will not work with 1.6.0.1 (no ast_dsp_set_threshold).

My users are already irked with me, so I can try some hacks to see
if it can be narrowed down.

A couple items that may help tuning the defaults-
1.  Pure VoIP conferences have no problem
2.  CNG may impact the issue

I notice the issue most dramatically on a Blackberry, but my
Windows Mobile phone sounds fine, with both devices calling
through the same H.323 gateway.


*** Quick update  ***
Changing the defined DEFAULT_THRESHOLD from 512 to 256 in main/dsp.
worked wonders.  I picked 256 based on the changes in trunk to
make it configurable, and it seems to be a sane value.


By: dea (dea) 2008-11-20 17:33:01.000-0600

I am still receiving reports of quality issues with the threshold set
to 256.  I tried lowering it to 128, but then I started to get
kernel panics.

Complaints about choppy audio trump complaints about conferences that
abort in the middle, so I am back to 256.

I cannot roll back to 1.4, so if there is anything I can do to help
move this along...
By: Russell Bryant (russell) 2008-11-21 07:13:28.000-0600

For 1.6.0, I think we should just make disabling it completely an option like it is in 1.4.  In addition to that, for future versions, having it optional and when it is enabled, having the ability to tweak the silence thresholds would be good.
By: Leif Madsen (lmadsen) 2008-11-21 08:00:20.000-0600

If that is the case, then why don't we just leave the option the same as it is now with a flag in MeetMe() so we don't have to break dialplans more than necessary?
By: dea (dea) 2008-11-21 11:03:06.000-0600

I would not mind if the option were turned off by default and we keep
the 1.4 'o' flag to turn it on.  I'd suggest that the threhold be a
config option stored in meetme.conf, since once a decent value is
establish for that system, it likely will not change, and it keeps
the dialplan a little cleaner.
By: dea (dea) 2008-11-21 11:37:25.000-0600

I was looking over the code in dsp.c and reviewing my long standing
system configuration, and I spotted something that may play into
this issue.

I have been using the global option silence_suppresssion for a long time.
I see that dsp.c reacts to that option very early in the code and returns
a NULL frame.

I will test later to see if dropping silence_suppression allows the
talker optimization to work better, with or without threshold tuning.
By: Tilghman Lesher (tilghman) 2008-11-24 15:21:35.000-0600

One more note.  From a DSP guy, he notes that the threshold may contain any value between 0 and 32767, inclusive.  So the fact that the value is initially an even exponent of 2 is not relevant to its functionality in this case.
By: dea (dea) 2008-11-24 15:44:11.000-0600

I am begining to think the issue is somewhere else in the code.

I have tried threshold set to 0,128 and 256.  There was some
improvement, but nothing major.

I started to suspect that maybe the extra RealTime scheduling
overhead was an issue, but a static conference with RealTime
disabled still sounds bad.  A conference of two or three callers
seems to work fine, but after the fourth caller, quality degrades
for that caller and beyond.  The first two/three callers do not
seem to be impacted.

I am running out of ideas to check, so maybe I will be rolling
back to 1.4
By: Leif Madsen (lmadsen) 2008-11-24 16:11:16.000-0600

Rolling back to Confirmed as it appears the attached patch doesn't seem to solve the issue so far :(
By: dea (dea) 2008-11-25 14:10:59.000-0600

No real progress, but another set of observations that may help-

1.  The quality seems to fall apart after the 3rd caller joins.
The first two callers heard clear audio, everyone else hears
the warble.
2.  1.6 now seems to mux in comfort noise/background noise.  1.4
produces crystal clear silence, even from noisy cell phones.  Maybe
the bridge/DSP is working to hard on useless audio.
3.  ztdummy/zttest score higher on accuracy than dahdi_dummy/dahdi_test

My environment does not have hardware timing source, only *dummy interfaces

I rolled the productio system back, but I have a test system that I can
continue to test possible fixes on.
By: dea (dea) 2008-11-26 13:23:47.000-0600

No real progress, but maybe another hint.  The issue
may be related to 12688.  I am no longer getting the
debug message about internal_timing, enabled or disabled.

The choppy audio may well be related to RTP timing not
being based on the internal timer and still trying to
base off incoming RTP...
By: dea (dea) 2008-12-03 18:51:51.000-0600

mdu113/justdave, can either of you share some details about your
environment?  I found a interesting bit while working on this
today.

During an RTP debug I spotted the fact (if the ts value can be
trusted) that we were sending 30ms of audio every 40ms.  The 30ms
was desired, but not at a 40ms interval.  I reconfigured my PSTN
gateways back to 20ms, and the quality improved dramatically.
By: mdu113 (mdu113) 2008-12-04 09:40:11.000-0600

Are you sure it's related to talker optimization? Have you checked what it sends with optimization disabled?
All my devices are (and always have been) configured for default 20ms per packet and so my opinion is based on the experience with that configuration.
Also, honestly I just don't believe in technology. It's based on the same principle as silence suppression/VAD and I've played quite some time with VAD implementation from different vendors several years ago and my conclusion is that if voice quality is of your concern you better disable it. I belive it just can't be made working fine for everybody. During that time I've been receiving constant complaints about comfort noise level is ok for one, but not good for another person complaining of dead silence, complaints of audible click when silence suppression kicks in and out, then there's always some inertia and more often than not when a person starts speaking first few milliseconds of the speech are cut out by silence suppression and so on.
When I tried to enable talker optimization feature all those complaints have immediately arose again, so I'd prefer to disable it and never look back. Bandwidth/computers are not that expensive today to risk customer satisfaction for the sake of that little savings.
By: Dave Miller (justdave) 2008-12-04 15:00:25.000-0600

meh, I wrote up stuff about my environment in here yesterday in response to DEA, but obviously didn't submit the form before closing the window or something. :(

Anyhow, in answer to mdu113, yes, it's definitely related to the talker optimization (or at least something triggered by it) because when I leave out the 'o' option to MeetMe, the problem doesn't happen at all.

As for our environment, the meetings in question where people notice this have calls coming in from all of ZAP, IAX2, and SIP, using an assortment of Polycom phones (both desk phones from offices and speakerphones in two physical conference rooms), softphone clients, cell phones, and landlines, typically about 20 to 40 phone connections on the call (about 80 or 100 individuals counting people physically present in the conference rooms).  Callers are typically auto-muted and have to dial *1 before they can talk.  The two physical conference rooms typically dial the *1 as soon as they join since they're conference rooms.  The problem has been observed both with people talking slightly too far away from the phone in a conference room and people on remote phones who seem to be talking normal, but it cuts them off frequently anyway.
By: Dave Miller (justdave) 2008-12-04 15:02:35.000-0600

And if I watch the output of MEETME LIST <confno> in a loop during the conference, the times people get cut off coincide with the "(talking)" indicator going away on their entry.
By: dea (dea) 2008-12-04 15:31:42.000-0600

OK, so I have been chasing a similar, but unrelated bug.  I still have some
test findings that might help this issue.  The biggest one I noted is
segfaults and a possible Kernel panic when I tried setting the detection
threshold to 0.

The comment added to the code recently about the threshold not needing to
be a power of 2 was a nice addition, but more helpful would be to identify
the units that threshold represents (ms, audio energy, something else) and
perhaps some suggestions on tuning (icrease or decrease if silence is
incorrectly detected)
By: dea (dea) 2009-02-05 16:47:56.000-0600

meetme-optimize-talker-off-trunk.txt is basically a revert of 62297

I have given up on trying to tune the threshold, it did not really make
any difference, other than segfaulting if I set it too low.

I'd suggest that the feature go back to being an 'opt-in' choice with
the tunable value, but my patch does not do that.
By: Digium Subversion (svnbot) 2009-02-18 13:12:49.000-0600

Repository: asterisk
Revision: 177101

U   trunk/apps/app_meetme.c

------------------------------------------------------------------------
r177101 | russell | 2009-02-18 13:12:49 -0600 (Wed, 18 Feb 2009) | 8 lines

Re-add 'o' option to MeetMe, reverting rev 62297.

Enabling this option by default proved to be a bad idea, as the talker detection
is not very reliable.  So, make it optional again, and off by default.

(issue ASTERISK-12981)
Reported by: justdave

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=177101
By: Digium Subversion (svnbot) 2009-02-18 13:25:24.000-0600

Repository: asterisk
Revision: 177135

_U  branches/1.6.0/
U   branches/1.6.0/apps/app_meetme.c

------------------------------------------------------------------------
r177135 | russell | 2009-02-18 13:25:24 -0600 (Wed, 18 Feb 2009) | 16 lines

Merged revisions 177101 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
r177101 | russell | 2009-02-18 13:12:49 -0600 (Wed, 18 Feb 2009) | 8 lines

Re-add 'o' option to MeetMe, reverting rev 62297.

Enabling this option by default proved to be a bad idea, as the talker detection
is not very reliable.  So, make it optional again, and off by default.

(issue ASTERISK-12981)
Reported by: justdave

........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=177135
By: Digium Subversion (svnbot) 2009-02-18 13:30:30.000-0600

Repository: asterisk
Revision: 177158

_U  branches/1.6.1/
U   branches/1.6.1/apps/app_meetme.c

------------------------------------------------------------------------
r177158 | russell | 2009-02-18 13:30:30 -0600 (Wed, 18 Feb 2009) | 16 lines

Merged revisions 177101 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
r177101 | russell | 2009-02-18 13:12:49 -0600 (Wed, 18 Feb 2009) | 8 lines

Re-add 'o' option to MeetMe, reverting rev 62297.

Enabling this option by default proved to be a bad idea, as the talker detection
is not very reliable.  So, make it optional again, and off by default.

(issue ASTERISK-12981)
Reported by: justdave

........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=177158
By: Russell Bryant (russell) 2009-02-20 14:00:36.000-0600

I made this optional again, and off by default.  Clearly, given the problems, it shouldn't be forced to be on.

If someone would like to work on improving it, and making it tweakable, that would be welcome.
By: Jason Parker (jparker) 2009-05-04 15:33:17

I believe this revert was incorrect, and broke the T option.

See attached patch (will be attached shortly).

Related to ASTERISK-14065
By: Jason Parker (jparker) 2009-05-05 17:15:34

Does the 13801-revisit.diff patch change the behavior of this issue at all?  It feels safe, but I'd like to make sure it doesn't break this again.
By: Digium Subversion (svnbot) 2009-05-14 12:05:43

Repository: asterisk
Revision: 194434

U   trunk/apps/app_meetme.c

------------------------------------------------------------------------
r194434 | file | 2009-05-14 12:05:40 -0500 (Thu, 14 May 2009) | 7 lines

Fix a bug where the 'T' option to Meetme did not work.

(closes issue ASTERISK-14065)
Reported by: Stochastic
(closes issue ASTERISK-12981)
Reported by: justdave

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=194434
By: Digium Subversion (svnbot) 2009-05-14 12:06:31

Repository: asterisk
Revision: 194435

_U  branches/1.6.0/
U   branches/1.6.0/apps/app_meetme.c

------------------------------------------------------------------------
r194435 | file | 2009-05-14 12:06:30 -0500 (Thu, 14 May 2009) | 14 lines

Merged revisions 194434 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r194434 | file | 2009-05-14 14:05:33 -0300 (Thu, 14 May 2009) | 7 lines
 
 Fix a bug where the 'T' option to Meetme did not work.
 
 (closes issue ASTERISK-14065)
 Reported by: Stochastic
 (closes issue ASTERISK-12981)
 Reported by: justdave
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=194435
By: Digium Subversion (svnbot) 2009-05-14 12:07:11

Repository: asterisk
Revision: 194436

_U  branches/1.6.1/
U   branches/1.6.1/apps/app_meetme.c

------------------------------------------------------------------------
r194436 | file | 2009-05-14 12:07:11 -0500 (Thu, 14 May 2009) | 14 lines

Merged revisions 194434 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r194434 | file | 2009-05-14 14:05:33 -0300 (Thu, 14 May 2009) | 7 lines
 
 Fix a bug where the 'T' option to Meetme did not work.
 
 (closes issue ASTERISK-14065)
 Reported by: Stochastic
 (closes issue ASTERISK-12981)
 Reported by: justdave
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=194436
By: Digium Subversion (svnbot) 2009-05-14 12:07:54

Repository: asterisk
Revision: 194437

_U  branches/1.6.2/
U   branches/1.6.2/apps/app_meetme.c

------------------------------------------------------------------------
r194437 | file | 2009-05-14 12:07:53 -0500 (Thu, 14 May 2009) | 14 lines

Merged revisions 194434 via svnmerge from
https://origsvn.digium.com/svn/asterisk/trunk

........
 r194434 | file | 2009-05-14 14:05:33 -0300 (Thu, 14 May 2009) | 7 lines
 
 Fix a bug where the 'T' option to Meetme did not work.
 
 (closes issue ASTERISK-14065)
 Reported by: Stochastic
 (closes issue ASTERISK-12981)
 Reported by: justdave
........

------------------------------------------------------------------------

http://svn.digium.com/view/asterisk?view=rev&revision=194437