[Home]

Summary:ASTERISK-27618: Crash occurs when sending a repeated number of INVITE messages over TCP or TLS transport
Reporter:Sandro Gauci (sandrogauci)Labels:patch pjsip security
Date Opened:2018-01-24 09:59:37.000-0600Date Closed:2018-02-21 10:38:05.000-0600
Priority:BlockerRegression?
Status:Closed/CompleteComponents:pjproject/pjsip
Versions:15.2.0 Frequency of
Occurrence
Related
Issues:
is duplicated byASTERISK-27471 res_pjsip_pubsub: Crash when accepting inbound subscription due to no memory pool
is related toASTERISK-27654 Crash in ast_sip_failover_request in PJSIP
Environment:Attachments:( 0) 1517596941-result.log
( 1) advisory.md
( 2) asterisk_menuselect+log.tar.gz
( 3) asterisk.log
( 4) asterisk-27618-15.patch
( 5) asterisk-config.tgz
( 6) build-config.tgz
Description:A crash occurs when a number of INVITE messages are sent over TCP or TLS and then the connection is suddenly closed. This issue leads to a segmentation fault.

Please see the attachment for full details and a script to reproduce.
Comments:By: Asterisk Team (asteriskteam) 2018-01-24 09:59:39.569-0600

This issue has been automatically restricted and set to a blocker due to being a security type issue. If this is not a security vulnerability issue it will be moved to the appropriate issue type when triaged.
By: Asterisk Team (asteriskteam) 2018-01-24 09:59:39.855-0600

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].
By: Sandro Gauci (sandrogauci) 2018-01-24 10:01:15.536-0600

Full details and code to reproduce the issue
By: George Joseph (gjoseph) 2018-01-24 11:03:33.674-0600

Hi Sandro,

"Tested versions: 15.2.0, 15.1.0, 15.0.0, 13.19.0, 13.11.2, 14.7.5"

Does the issue happen on all the versions you tested?

By: Sandro Gauci (sandrogauci) 2018-01-24 12:19:14.391-0600

yes
By: George Joseph (gjoseph) 2018-01-24 14:25:17.687-0600

I can't seem to reproduce this on either Asterisk 13 or 15.  How long does it take before you get the crash?

Can I get the entire contents of the /etc/asterisk directory, and from the build, config.log, makeopts and menuselect.makeopts?



By: Sandro Gauci (sandrogauci) 2018-01-24 23:55:42.168-0600

attached configuration files
By: George Joseph (gjoseph) 2018-01-25 10:53:55.622-0600

Thanks [~sandrogauci],  I can reproduce now and work is in progress.

By: George Joseph (gjoseph) 2018-01-30 14:29:59.837-0600

So, we've got some code fixes in the works that can help with this issue but they don't solve it.  We're asking the guys at Teluu for help fixing the root cause in pjproject.

By: Sandro Gauci (sandrogauci) 2018-01-30 23:04:48.008-0600

Thanks for the update
By: George Joseph (gjoseph) 2018-02-02 09:11:58.997-0600

[~sandrogauci], can you repeat your tests with the res_pjsip_transport_management module loaded?

By: Sandro Gauci (sandrogauci) 2018-02-02 09:34:46.545-0600

Sure - will test and let you know
By: Sandro Gauci (sandrogauci) 2018-02-02 10:00:13.438-0600

Made sure to add that to the modules.conf and checked that it is loaded:
```
Loading res_pjsip_transport_management.so.
 == res_pjsip_transport_management.so => (PJSIP Reliable Transport Management)
```

Then ran the script and observed the Asterisk process crashing:

```
*CLI>   == Setting global variable 'SIPDOMAIN' to '127.0.0.1'
   -- Executing [3000@Dial-Users:1] Verbose("PJSIP/3000-00000000", "1, "User 3000 dialed 3000."") in new stack
 "User 3000 dialed 3000."
   -- Executing [3000@Dial-Users:2] Set("PJSIP/3000-00000000", "SAC_DIALED_EXTEN=3000") in new stack
   -- Executing [3000@Dial-Users:3] GotoIf("PJSIP/3000-00000000", "0?dialed-BUSY,1:") in new stack
   -- Executing [3000@Dial-Users:4] Dial("PJSIP/3000-00000000", "PJSIP/3000,30") in new stack
 == Everyone is busy/congested at this time (1:0/0/1)
   -- Executing [3000@Dial-Users:5] Goto("PJSIP/3000-00000000", "dialed-CHANUNAVAIL,1") in new stack
   -- Goto (Dial-Users,dialed-CHANUNAVAIL,1)
   -- Executing [dialed-CHANUNAVAIL@Dial-Users:1] NoOp("PJSIP/3000-00000000", "") in new stack
   -- Executing [dialed-CHANUNAVAIL@Dial-Users:2] Playback("PJSIP/3000-00000000", "pbx-invalid") in new stack
 == Setting global variable 'SIPDOMAIN' to '127.0.0.1'
./buildasterisk.sh: line 60:    14 Segmentation fault      (core dumped) /opt/asterisk/sbin/asterisk -fcvvvvv
```

This is the output from the test tool:

```
python test.py
Authorization: Digest username="3000",realm="asterisk",nonce="1517587073/7d9124586f780ec8263c4045cb6bc2a6",uri="sip:127.0.0.1:5061",response="cf4a90f3f2c39fca5bef6239c5ffb949",algorithm=md5

EOF occurred in violation of protocol (_ssl.c:590)
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
[Errno 111] Connection refused
getting close!
confirmed dead
```

Are you having trouble reproducing this one?
By: George Joseph (gjoseph) 2018-02-02 10:32:07.603-0600

I can reproduce it without res_pjsip_transport_management loaded but I get no crashes with that module loaded.

Can you get me a backtrace of a crash with that module loaded, and can you get me menuselect.makeopts (as opposed to menuselect/makeopts)?

By: George Joseph (gjoseph) 2018-02-02 10:37:44.106-0600

Also, how about a log with ERROR,WARNING,VERBOSE,NOTICE turned on?

By: Sandro Gauci (sandrogauci) 2018-02-02 12:48:31.691-0600

There you go.
By: Sandro Gauci (sandrogauci) 2018-02-04 02:36:57.676-0600

Requested asterisk and gdb logs and menuselect file
By: George Joseph (gjoseph) 2018-02-05 07:48:05.379-0600

Thanks Sandro.  I can now reproduce the crash on 15.2.0.  When you get a chance, can you retest 13.19.0 with res_pjsip_transport_management loaded?

By: George Joseph (gjoseph) 2018-02-05 16:53:12.405-0600

No need to test 13.19.

Attached is a patch for asterisk-15 that should resolve the issue completely.  Can you test and confirm?
It won't apply to asterisk 13 but I'll have one for 13 shortly.

By: Sandro Gauci (sandrogauci) 2018-02-06 07:54:01.478-0600

thanks! Will test and let you know.
By: Sandro Gauci (sandrogauci) 2018-02-10 00:06:18.717-0600

Tested the patch and it looks like the issue has been fixed. Thanks!
By: Friendly Automation (friendly-automation) 2018-02-21 10:38:07.210-0600

Change 8323 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8323|https://gerrit.asterisk.org/8323]
By: Friendly Automation (friendly-automation) 2018-02-21 10:38:13.253-0600

Change 8324 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8324|https://gerrit.asterisk.org/8324]
By: Friendly Automation (friendly-automation) 2018-02-21 10:38:54.275-0600

Change 8325 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8325|https://gerrit.asterisk.org/8325]
By: Friendly Automation (friendly-automation) 2018-02-21 10:39:05.931-0600

Change 8326 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8326|https://gerrit.asterisk.org/8326]
By: Friendly Automation (friendly-automation) 2018-02-21 10:39:22.942-0600

Change 8328 merged by George Joseph:
AST-2018-005: Add a check for NULL tdata in ast_sip_failover_request

[https://gerrit.asterisk.org/8328|https://gerrit.asterisk.org/8328]
By: Friendly Automation (friendly-automation) 2018-02-21 10:39:32.658-0600

Change 8329 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8329|https://gerrit.asterisk.org/8329]
By: Friendly Automation (friendly-automation) 2018-02-21 10:39:42.767-0600

Change 8330 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8330|https://gerrit.asterisk.org/8330]
By: Friendly Automation (friendly-automation) 2018-02-21 10:40:33.105-0600

Change 8331 merged by George Joseph:
AST-2018-005: Add a check for NULL tdata in ast_sip_failover_request

[https://gerrit.asterisk.org/8331|https://gerrit.asterisk.org/8331]
By: Friendly Automation (friendly-automation) 2018-02-21 10:40:48.684-0600

Change 8332 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8332|https://gerrit.asterisk.org/8332]
By: Friendly Automation (friendly-automation) 2018-02-21 10:40:58.171-0600

Change 8333 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8333|https://gerrit.asterisk.org/8333]
By: Friendly Automation (friendly-automation) 2018-02-21 10:41:57.924-0600

Change 8334 merged by George Joseph:
AST-2018-005: Add a check for NULL tdata in ast_sip_failover_request

[https://gerrit.asterisk.org/8334|https://gerrit.asterisk.org/8334]
By: Friendly Automation (friendly-automation) 2018-02-21 10:42:06.312-0600

Change 8335 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8335|https://gerrit.asterisk.org/8335]
By: Friendly Automation (friendly-automation) 2018-02-21 10:42:16.692-0600

Change 8336 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8336|https://gerrit.asterisk.org/8336]
By: Friendly Automation (friendly-automation) 2018-02-21 10:42:34.279-0600

Change 8337 merged by George Joseph:
AST-2018-005: Add a check for NULL tdata in ast_sip_failover_request

[https://gerrit.asterisk.org/8337|https://gerrit.asterisk.org/8337]
By: Friendly Automation (friendly-automation) 2018-02-21 10:42:42.715-0600

Change 8338 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8338|https://gerrit.asterisk.org/8338]
By: Friendly Automation (friendly-automation) 2018-02-21 10:42:51.296-0600

Change 8339 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8339|https://gerrit.asterisk.org/8339]
By: Friendly Automation (friendly-automation) 2018-02-21 10:43:09.855-0600

Change 8340 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8340|https://gerrit.asterisk.org/8340]
By: Friendly Automation (friendly-automation) 2018-02-21 10:43:18.689-0600

Change 8341 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8341|https://gerrit.asterisk.org/8341]
By: Friendly Automation (friendly-automation) 2018-02-21 10:43:33.535-0600

Change 8342 merged by George Joseph:
AST-2018-005: Add a check for NULL tdata in ast_sip_failover_request

[https://gerrit.asterisk.org/8342|https://gerrit.asterisk.org/8342]
By: Friendly Automation (friendly-automation) 2018-02-21 10:43:45.150-0600

Change 8343 merged by George Joseph:
AST-2018-005: Fix tdata leaks when calling pjsip_endpt_send_response(2)

[https://gerrit.asterisk.org/8343|https://gerrit.asterisk.org/8343]
By: Friendly Automation (friendly-automation) 2018-02-21 10:43:54.333-0600

Change 8344 merged by George Joseph:
AST-2018-005: res_pjsip_transport_management:  Move to core

[https://gerrit.asterisk.org/8344|https://gerrit.asterisk.org/8344]