Summary:	ASTERISK-19678: Manager disconnects on return of large dataset with action_command
Reporter:	mbrevda (lazytt)	Labels:
Date Opened:	2012-04-09 06:38:28	Date Closed:	2012-05-23 10:45:51
Priority:	Major	Regression?	No
Status:	Closed/Complete	Components:	Core/ManagerInterface
Versions:	1.8.5.0	Frequency of Occurrence	Constant
Related Issues:
Environment:	Linux localhost.localdomain 2.6.18-194.el5PAE #1 SMP Fri Apr 2 15:37:44 EDT 2010 i686 i686 i386 GNU/Linux (CentOS)	Attachments:	( 0) chatlog-asterisk-dev.txt
Description:	Requesting data from the manager for a cli command crashes the manager connection when dataset returned is large. The maximum amount of data returned is variable, but seems seems to sit between 3.2mb and 3.9mb. This behavior has been experience both via telnet and via a php socket, seemingly exonerating the client. Additional data: When making a manager request for cli data (for example: Action: Command\nCommand: database show), Asterisk first outputs the data to a temporary file. This file has been verified to be written correctly with all data intact. The issue is when replaying the data back to the client (i.e. reading out the file).
Comments:	By: mbrevda (lazytt) 2012-04-09 06:45:33.445-0500 attaching chat log, in case there is any interesting data in there By: Matt Jordan (mjordan) 2012-04-09 09:49:39.428-0500 Thank you for your bug report. In order to move your issue forward, we require a backtrace[1] from the core file produced after the crash. Also, be sure you have DONT_OPTIMIZE enabled in menuselect within the Compiler Flags section, then: make install After enabling, reproduce the crash, and then execute the backtrace[1] instructions. When complete, attach that file to this issue report. [1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace By: mbrevda (lazytt) 2012-04-10 10:10:22.206-0500 Matt, As per our IRC chat, there doesn't seem to be a need for a backtrace as asterisk isn't crashing - just the manager connection is. Whats the next step here? By: Matt Jordan (mjordan) 2012-04-17 08:42:05.717-0500 I don't think there's any other action for you to take at this time.  We'll move this forward in the process. By: mbrevda (lazytt) 2012-05-21 15:41:05.077-0500 For reference: this issue, while trivial in some settings, is CRUCIAL for systems running FreePBX. FreePBX relies heavily on the astDB for many settings (Basically, anything that the user (i.e. not the sysadmin) might want to be able to change, or anything that is changeable from a phone (fm, cf, etc) goes there) and the Backup & Restore module backups up the astDB. Users were surprised to find that there backups weren't being restored 100%, until we traced the issue down to the AMI not outputting all the data to the backup (when the backup is created). Here is one such ticket: http://www.freepbx.org/trac/ticket/5839 (a few others were submitted on non-public ticketing systems). Thanks! By: Philippe Lindheimer (p_lindheimer) 2012-05-21 15:45:28.888-0500 to further comment on mbrevda's point, the database_show() command is used within the asterisk manager php library to pull much of this information, although this issue has been mostly seen on corrupt restores, where it is too late as the data is already lost, it is accessed any time the database is regenerated and that can easily result in silent data corruption since dialplan generation depends on proper settings of the data stored there. By: mbrevda (lazytt) 2012-05-23 10:45:13.157-0500 Thanks a lot to @qwell for pointing out https://issues.asterisk.org/jira/browse/ASTERISK-19603, which is a similar issue. Setting writetimeout to something higher seems to keep things running smoothly - and resolves the issue presented here. By: mbrevda (lazytt) 2012-05-23 10:45:51.829-0500 Up-ing the value of writetimeout in manager.conf resolves this issue. Thanks all!