| Summary: | ASTERISK-22805: res_rtp_asterisk: Crash when calling BIO_ctrl_pending in dtls_srtp_check_pending when dialed by JSSIP | ||||
| Reporter: | Dmitry Burilov (netaskd) | Labels: | |||
| Date Opened: | 2013-11-01 09:56:51 | Date Closed: | 2015-07-07 14:56:55 | ||
| Priority: | Critical | Regression? | |||
| Status: | Closed/Complete | Components: | Channels/chan_sip/General Resources/res_rtp_asterisk | ||
| Versions: | 11.5.1 11.6.0 11.7.0 | Frequency of Occurrence | |||
| Related Issues: |
| ||||
| Environment: | Linux 2.6.32-358.18.1.el6.x86_64, OpenSSL 1.0.1e-fips 11 Feb 2013, srtp 1.4.4 | Attachments: | ( 0) backtrace.txt ( 1) backtrace2.txt ( 2) bt_udptl.txt ( 3) coredump.tar.bz2 ( 4) sip.conf | ||
| Description: | Dial from Chrome 30.0.1599.101m via jssip application crash asterisk11.
-----------gdb output ---------------- Core was generated by `/usr/sbin/asterisk -f -vvvg -c'. Program terminated with signal 11, Segmentation fault. #0 0x00007f1353750875 in BIO_ctrl (b=0x7f132403bd80, cmd=10, larg=0, parg=0x0) at bio_lib.c:367 367 ((ret=cb(b,BIO_CB_CTRL,parg,cmd,larg,1L)) <= 0)) ---------------------------- ----------- and ------------ #0 BIO_ctrl (b=0x7f824404ec30, cmd=10, larg=0, parg=0x0) at bio_lib.c:370 370 ret=b->method->ctrl(b,cmd,larg,parg); [?1034h(gdb) frame 1 #1 0x00007f82d3ac3702 in dtls_srtp_check_pending (instance=0x7f824403e158, rtp=0x7f8244043360) at res_rtp_asterisk.c:1258 1258 size_t pending = BIO_ctrl_pending(rtp->write_bio); (gdb) info frame 1 Stack frame at 0x7f82cc914dd0: rip = 0x7f82d3ac3702 in dtls_srtp_check_pending (res_rtp_asterisk.c:1258); saved rip 0x7f82d3ac40e6 called by frame at 0x7f82cc914f20, caller of frame at 0x7f82cc914cd0 source language c. Arglist at 0x7f82cc914dc0, args: instance=0x7f824403e158, rtp=0x7f8244043360 Locals at 0x7f82cc914dc0, Previous frame's sp is 0x7f82cc914dd0 Saved registers: rbx at 0x7f82cc914db0, rbp at 0x7f82cc914dc0, r12 at 0x7f82cc914db8, rip at 0x7f82cc914dc8 ---------------------------- | ||||
| Comments: | By: Matt Jordan (mjordan) 2013-11-01 10:14:50.902-0500 Thank you for your bug report. In order to move your issue forward, we require a backtrace[1] from the core file produced after the crash. Also, be sure you have DONT_OPTIMIZE enabled in menuselect within the Compiler Flags section, then: make install After enabling, reproduce the crash, and then execute the backtrace[1] instructions. When complete, attach that file to this issue report. [1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace That is not a complete backtrace. Please follow the instructions on the Asterisk wiki and attach a full, complete backtrace. By: Dmitry Burilov (netaskd) 2013-11-01 10:40:18.983-0500 I create backtrace. In attache. By: Matt Jordan (mjordan) 2013-11-01 10:49:04.767-0500 Your backtrace appears to come from an un-optimized version of Asterisk: {noformat} #1 0x00007f12f34daa14 in dtls_srtp_check_pending (instance=0x7f132402b7d8, rtp=<value optimized out>) at res_rtp_asterisk.c:1244 pending = <value optimized out> {noformat} Are you sure you compiled Asterisk with DONT_OPTIMIZE? That aside - based on the backtrace, it looks like we're crashing when calling BIO_ctrl_pending: {noformat} size_t pending = BIO_ctrl_pending(rtp->write_bio); {noformat} If you have the core file, it'd be very interesting to see what the value of {{rtp->write_bio}} is. Can you attach your {{sip.conf}} to this issue as well? Did you configure {{sip.conf}} to support DTLS-SRTP? By: Dmitry Burilov (netaskd) 2013-11-01 11:15:58.386-0500 if "MENUSELECT_CFLAGS=DONT_OPTIMIZE LOADABLE_MODULES", then DONT_OPTIMIZE enabled or disabled? coredump and sip.conf attached. By: Matt Jordan (mjordan) 2013-11-04 11:35:38.620-0600 Unfortunately, we can't actually investigate the core file. Core files are only useful on the machine where they were created - anywhere else, it's just gibberish. If you can, however, it'd be useful to know what {{rtp->write_bio}} is in stack frame 1. GDB can open the core file and provide you that information. By: Dmitry Burilov (netaskd) 2013-11-06 08:26:09.615-0600 rtp value By: Dmitry Burilov (netaskd) 2013-11-06 08:29:13.212-0600 I attached new backtrace2.txt file with enabled DONT_OPTIMIZE flags, and now you can see rtp value. By: JoshE (n8ideas) 2014-05-29 14:23:13.767-0500 This may not be 100% related to this, as I have been messing with ASTERISK-22961, but I've also found this crash path. This is on sending an outbound call from Asterisk to an off premise analog fax machine with faxdetect=yes on the peer. Is 100% reproducible at this point. By: Fidel Gonzalez (fgvazquez) 2015-02-05 10:37:19.036-0600 Hi I have the same crash; Dial from Chrome <Version 40.0.2214.94 m> using SIPML5 demo <Local installation > Note: In order to complete the call in the sipml5 demo we modified the file <tmedia_session_jsep.js> adding: 'optional': [{DtlsSrtpKeyAgreement: false}] ======Asterisk 11.0.1 + patch============ {noformat} (gdb) bt #0 0x00007f86c8000138 in ?? () #1 0x00007f86d4d54399 in BIO_read () from /usr/lib64/libcrypto.so.10 #2 0x00007f8693c845af in dtls_srtp_check_pending (instance=0x7f86c80508b8, rtp=0x7f86c8054e00) at res_rtp_asterisk.c:1231 #3 0x00007f8693c84df4 in __rtp_recvfrom (instance=0x7f86c80508b8, buf=0x7f86c8054fb8, size=8192, flags=0, sa=0x7f8689cc6c60, rtcp=0) at res_rtp_asterisk.c:1389 #4 0x00007f8693c851f8 in rtp_recvfrom (instance=0x7f86c80508b8, buf=0x7f86c8054fb8, size=8192, flags=0, sa=0x7f8689cc6c60) at res_rtp_asterisk.c:1475 ....................................................................................... (gdb) f 2 #2 0x00007f8693c845af in dtls_srtp_check_pending (instance=0x7f86c80508b8, rtp=0x7f86c8054e00) at res_rtp_asterisk.c:1231 1231 out = BIO_read(rtp->write_bio, outgoing, sizeof(outgoing)); (gdb) p * rtp->write_bio $1 = {method = 0x7f86c8010190, callback = 0, cb_arg = 0x0, init = 1, shutdown = 1, flags = 0, retry_reason = 0, num = -1, ptr = 0x0, next_bio = 0x0, prev_bio = 0x0, references = 0, num_read = 0, num_write = 0, ex_data = {sk = 0x0, dummy = 0}} (gdb) bt full #2 0x00007f8693c845af in dtls_srtp_check_pending (instance=0x7f86c80508b8, rtp=0x7f86c8054e00) at res_rtp_asterisk.c:1231 outgoing = warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 0x7f8689cc6450 "port" out = 140727337391 remote_address = {ss = {ss_family = 2, __ss_align = 0, __ss_padding = '\000' <repeats 111 times>}, len = 16} ice = 32646 pending = 18446744073709551614 #3 0x00007f8693c84df4 in __rtp_recvfrom (instance=0x7f86c80508b8, buf=0x7f86c8054fb8, size=8192, flags=0, sa=0x7f8689cc6c60, rtcp=0) at res_rtp_asterisk.c:1389 in = 0x7f86c8054fb8 "" len = 132 rtp = 0x7f86c8054e00 srtp = 0x7f86c8042c10 __PRETTY_FUNCTION__ = "__rtp_recvfrom" {noformat} =========== Asterisk 11.5.0 ================= {noformat} (gdb) bt #0 0x00007f972401f8a0 in ?? () #1 0x00007f974567d37f in BIO_read () from /usr/lib64/libcrypto.so.10 #2 0x00007f970561e638 in dtls_srtp_check_pending (instance=0x7f97240454b8, rtp=0x7f972404a6c0) at res_rtp_asterisk.c:1254 #3 0x00007f970561ee7d in __rtp_recvfrom (instance=0x7f97240454b8, buf=0x7f972404a878, size=8192, flags=0, sa=0x7f96fb760c70, rtcp=0) at res_rtp_asterisk.c:1411 #4 0x00007f970561f297 in rtp_recvfrom (instance=0x7f97240454b8, buf=0x7f972404a878, size=8192, flags=0, sa=0x7f96fb760c70) at res_rtp_asterisk.c:1497 (gdb) f 2 #2 0x00007f970561e638 in dtls_srtp_check_pending (instance=0x7f97240454b8, rtp=0x7f972404a6c0) at res_rtp_asterisk.c:1254 1254 out = BIO_read(rtp->write_bio, outgoing, sizeof(outgoing)); (gdb) p * rtp->write_bio $1 = {method = 0x7f9724035bd0, callback = 0x7f972401f8a0, cb_arg = 0x0, init = 1, shutdown = 1, flags = 0, retry_reason = 0, num = -1, ptr = 0x0, next_bio = 0x0, prev_bio = 0x0, references = 0, num_read = 0, num_write = 0, ex_data = { sk = 0x0, dummy = 1423153344}} (gdb) (gdb) bt full #0 0x00007f972401f8a0 in ?? () No symbol table info available. #1 0x00007f974567d37f in BIO_read () from /usr/lib64/libcrypto.so.10 No symbol table info available. #2 0x00007f970561e638 in dtls_srtp_check_pending (instance=0x7f97240454b8, rtp=0x7f972404a6c0) at res_rtp_asterisk.c:1254 outgoing = warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 warning: Range for type (null) has invalid bounds 0..-3 0x7f96fb760470 "\363%,\020\227\177" out = 138623816768 remote_address = {ss = {ss_family = 2, __ss_align = 0, __ss_padding = '\000' <repeats 111 times>}, len = 16} ice = 32662 pending = 18446744073709551614 {noformat} By: Joshua C. Colp (jcolp) 2015-07-06 06:00:14.151-0500 A change is now up for review at the following addresses for a fix to this problem. While our code review process is pretty fast these days if anyone would like to test the change and provide feedback on this issue it would be welcome: 11: https://gerrit.asterisk.org/#/c/786/ 13: https://gerrit.asterisk.org/#/c/787/ master: https://gerrit.asterisk.org/#/c/788/ The patch can be downloaded by clicking the "Download" dropdown and selecting the method you wish. | ||||