Summary:	ASTERISK-05571: Ztdummy w/ RTC Support Degrades Audio
Reporter:	damin (damin)	Labels:
Date Opened:	2005-11-11 18:08:11.000-0600	Date Closed:	2011-06-07 14:10:20
Priority:	Minor	Regression?	No
Status:	Closed/Complete	Components:	Core/General
Versions:		Frequency of Occurrence
Related Issues:
Environment:		Attachments:
Description:	With Ztdummy loaded, audio crackles and pops ONLY when RTC on CPU0 is interrupting. When the RTC interrupt load moves to CPU1, the audio is fine. I want to chalk this up to a Hardware issue, but we would like to verify that this behavior is not reproducible on other systems. Justinu is going to test on his Dual Processor Box under the same conditions and report back. ****** STEPS TO REPRODUCE ****** 1. Add "#define USE_RTC" to zconfig.h 2. Rebuild Zaptel 3. Load ztdummy module 4. Call to an extension that plays musiconhold 5. Monitor interrupts and listen to the audio w/ the following: watch -n1 'cat /proc/interrupts' Take note of if/when audio pops/crackles occur. See if they happen on one, or both of the CPUS. If audio is fine, then call me a lunatic. If not, remove ztdummy module and repeat to verify that the Audio cleans up. ****** ADDITIONAL INFORMATION ****** I am building a new test-box for a client that will eventually end up being a Call Queue and Conferencing server for SIP and Zap channels. This is a 512 Meg, Dual P-III 700 box running an Intel board. It is strictly a test box for now. Installed Asterisk from CVS-Head as of 16:00 EST on 2005-11-11. Since we do not yet have the TE-405P for this board, on KPF's suggestion, I enabled USE_RTC and compiled Ztdummy.
Comments:	By: damin (damin) 2005-11-11 18:39:55.000-0600 <justinu> damin: testing now <justinu> damin: confirmed <justinu> happens on my box too <Damin> justinu: REally? <justinu> yep, lemme add my findings to the ticket <Damin> justinu: Cpu0 = borked, cpu1 = fine? <justinu> damin: yes, same exact behavior <bweschke_> the question then is .............. is RTC hosed on SMP kernels? <bweschke_> and asterisk is just inheriting the bad behavior by using it? <bweschke_> because if u disable RTC and go back to OHCI / USB on ztdummy, I bet you this goes away too, no? Hmm.. well I rebuilt and tested w/ RTC disabled, and the cracking and popping is definitely much less noticeable. I've been listening to hold music for 2 minutes now, and only heard one small pop. So this does seem to be related to the use of the Real-Time Clock. KP Flemming asked me if the Ethernet card for this machine was using CPU0 for interrupts and the answer was "Yes". By: Kevin P. Fleming (kpfleming) 2005-11-11 19:20:42.000-0600 Changing to 'minor' because there are reasonable workarounds available... By: Justin Unger (justinu) 2005-11-12 11:51:51.000-0600 I can confirm I experienced the same behavior. I compiled CVS HEAD from aprox 16:00PST 11/11/05, running with ztdummy, using the same compile options as Damin, on an SMP machine. Whenever you listen to Music on Hold, the audio is corrupted whenever the RTC is interrupting CPU0. As soon as it switches to CPU1, the audio is fine. When it switches back to CPU0, you get corrupted audio from Music On Hold. Some hardware details: Dell PowerEdge 2450, 2xP3-600, CentOS 4.2, Kernel 2.6.9-22.ELsmp, no zaptel hardware installed. By: Justin Unger (justinu) 2005-11-12 11:59:15.000-0600 Switching from native MoH using format_mp3 to using mpg123 seems to make the problem worse. By: Alberto Sagredo (albersag) 2005-11-12 14:47:46.000-0600 Maybe zaptel drivers were not compiled with rtc?. Is rtc on your kernel as module or compiled in in?. I had some problems with this some weeks ago with 1.09 an zaptel 1.09 p2 on Gentoo. By: Mark Spencer (markster) 2005-11-12 15:10:10.000-0600 This is pretty clearly a hardware / kernel issue, and I don't think it belongs in the bug tracker.  Maybe someone can put a wiki entry about how to make it run on CPU 1? By: damin (damin) 2005-11-12 17:51:28.000-0600 Justinu: Is your Ethernet card also using interrupts on CPU0? albersag: My kernel is a stock Centos 4.2 kernel. I believe the RTC is loaded as a module. Also, Zaptel was compiled w/ "#define USE_RTC" in zconfig.h. It's definitely using the RTC. Markster: I'd respectfully submit that since this is reproducible on two systems, each w/ different hardware, that this is a software issue, not a hardware issue. Wether it is Linux's RTC implementation or Ztdummy's fault has yet to be determined. My only purpose in opening this as a bug was to note the problem, and see if I can get corroboration from others that they experience the same behavior. As of tonight, we know the problem occurs under the following conditions: 1. #define USE_RTC in zconfig.h 2. Ztdummy loaded 3. Native MOH as well as mpg123 exhibit the problem 4. The problem is only apparent on Dual Processor machines and only when the RTC is interrupting on CPU0. CPU1 is fine. I will research the rtc module under Linux to see if there are load-time options that let you control how interrupts are distributed between CPUs in an SMP kernel. I also realize that given the other issues on the table, this is a relatively minor issue and that since zaptel timing seems to work fine w/ Digium hardware, it probably isn't high on the priority list to fix it. I would just request that we stay concious of the fact that this is a software issue, and most likely fixable.