Summary: | ASTERISK-29203: res_pjsip_t38: Crash when changing state | ||
Reporter: | Gregory Massel (gmza) | Labels: | fax patch security |
Date Opened: | 2020-12-08 08:32:37.000-0600 | Date Closed: | 2021-02-18 10:40:08.000-0600 |
Priority: | Major | Regression? | Yes |
Status: | Closed/Complete | Components: | Resources/res_pjsip_session Resources/res_pjsip_t38 |
Versions: | 16.13.0 16.14.0 16.15.0 16.16.0 | Frequency of Occurrence | Frequent |
Related Issues: | |||
Environment: | Ubuntu 18.04.5 LTS, kernel 5.3.0, Asterisk 16.16.0. | Attachments: | ( 0) AST-2021-002.pdf ( 1) ASTERISK-29203.diff ( 2) call1.pcap ( 3) call2.pcap ( 4) call2-cpe-leg.pcap ( 5) core-brief.txt-vpbx0-2020-12-08-11h04 ( 6) core-brief.txt-vpbx1-2020-12-08-15h32 ( 7) core-brief.txt-vpbx5-2020-12-08-13h51 ( 8) core-full.txt-vpbx0-2020-12-08-11h04 ( 9) core-full.txt-vpbx1-2020-12-08-15h32 (10) core-full.txt-vpbx5-2020-12-08-13h51 (11) core-info.txt-vpbx0-2020-12-08-11h04 (12) core-info.txt-vpbx1-2020-12-08-15h32 (13) core-info.txt-vpbx5-2020-12-08-13h51 (14) core-locks.txt-vpbx0-2020-12-08-11h04 (15) core-locks.txt-vpbx1-2020-12-08-15h32 (16) core-locks.txt-vpbx5-2020-12-08-13h51 (17) core-thread1.txt-vpbx0-2020-12-08-11h04 (18) core-thread1.txt-vpbx1-2020-12-08-15h32 (19) core-thread1.txt-vpbx5-2020-12-08-13h51 (20) crashed_asterisk-to-upstream_asterisk_switch.pcap (21) phone-to-asterisk-via-proxy.pcap (22) sip_trace_2.png (23) sip_trace.png (24) verbose-log |
Description: | Three different systems running Asterisk 16.15.0 have segfaulted today. Coredumps show the exact same issue within t38_change_state.
The only recent change has been upgrading from 16.12.0 to 16.15.0 (which was done 6 days ago) hence it would appear that this bug was introduced after 16.12.0. | ||
Comments: | By: Asterisk Team (asteriskteam) 2020-12-08 08:32:40.258-0600 Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed. A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report. Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process]. Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur. Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/]. By: Gregory Massel (gmza) 2020-12-08 08:34:36.228-0600 Core dump analysis attached. Note the actual core files are approximately 2GB each and have therefore been excluded. By: Joshua C. Colp (jcolp) 2020-12-08 08:41:17.493-0600 Do you have SIP traces and console output to go with this? By: Gregory Massel (gmza) 2020-12-08 09:04:40.934-0600 There is no console output as it crashes before logging anything, however, I will pull the SIP traces and PCAPs and attach them. This should be within the next hour or two. By: Gregory Massel (gmza) 2020-12-08 12:25:17.973-0600 trace and pcap of one of the calls attached By: Gregory Massel (gmza) 2020-12-08 12:26:49.341-0600 [Dec 8 13:51:02] WARNING[5003] udptl.c: UDPTL (no tag): Cannot calculate local_max_datagram before local_max_ifp has been set. By: Gregory Massel (gmza) 2020-12-08 13:27:08.952-0600 Here is the trigger call from another crash. console output: [Dec 8 15:32:13] WARNING[24197] udptl.c: UDPTL (no tag): Cannot calculate local_max_datagram before local_max_ifp has been set. By: Joshua C. Colp (jcolp) 2020-12-09 04:12:31.694-0600 Has this happened multiple times since? As well by console output I mean the logging mentioned on the wiki[1]. [1] https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information By: Gregory Massel (gmza) 2020-12-09 08:02:05.341-0600 I downgraded to 16.12.0 last night and the issue has not repeated since the downgrade. The logs have HUGE amounts of unrelated information. Shortly after the two legs bridge, a T.38 re-INVITE ocurrs and the only relevant log entries (i.e. that don't pertain to other calls) are: WARNING[24197] udptl.c: UDPTL (no tag): Cannot calculate local_max_datagram before local_max_ifp has been set. In two of the three cases, that was the last log entry. In the third, there were still some log entries for unrelated calls that flushed to the logs after that. By: Joshua C. Colp (jcolp) 2020-12-09 08:10:36.915-0600 Okay, have you filtered the pcaps? I'm seeing a lack of SIP packets for some transactions, including those doing the SDP negotiation resulting in not having a complete picture. By: Joshua C. Colp (jcolp) 2020-12-09 08:13:01.505-0600 And what is 196.216.192.4, and if Asterisk what version was it running? By: Gregory Massel (gmza) 2020-12-09 08:26:53.044-0600 I didn't filter the PCAPs; they were pulled using VoIPmonitor (voipmonitor.org). If I look at the pcaps in Wireshark, I see the full flow. 196.216.192.4 is Asterisk 16.11.1 with the patch for ASTERISK-28900 (as found in versions 16.12.0 onwards). Unfortunately I cannot upgrade that box to a newer version until ASTERISK-29035 is resolved. 196.216.192.5 is OpenSIPS + RTPengine. By: Joshua C. Colp (jcolp) 2020-12-09 08:47:50.199-0600 If I look at call2.pcap and both manually examine or use Telephony -> VoIP Calls I see SIP requests, but not SIP responses except for one at the end. By: Gregory Massel (gmza) 2020-12-09 11:14:37.576-0600 That is because Asterisk segfaults.... so it never responds and the other box retries the message a few times until giving up. By: Joshua C. Colp (jcolp) 2020-12-09 11:16:06.441-0600 I don't see ANY SIP response except for a single 481 at the end, including ones for SDP negotiation. There must have been ones - because there are ACKs for the 200 OKs. By: Joshua C. Colp (jcolp) 2020-12-09 11:17:17.881-0600 Why I'm asking for these is because with all the SIP traffic including SDP a SIPp scenario can be crafted which matches this scenario as closely as possible. By: Gregory Massel (gmza) 2020-12-09 11:18:09.709-0600 Sorry, for clarity, 196.216.192.4 is NOT segfaulting. It's 196.216.192.3 and 196.216.192.9 that segfaulted in these scenarios. By: Gregory Massel (gmza) 2020-12-09 11:22:49.545-0600 It both cases, the call sets up and negotiates G.711 A-law audio correctly. Then, shortly after connecting, 196.216.192.4 sends an in-dialog re-INVITE with T.38 in the SDP. The other Asterisk box receives that packet, logs "WARNING[5003] udptl.c: UDPTL (no tag): Cannot calculate local_max_datagram before local_max_ifp has been set." and segfaults; it never responses to the re-INVITE. By: Gregory Massel (gmza) 2020-12-09 11:26:16.424-0600 That 481 is a red herring. If you look at the time involved before it sends that, what has actually happened is that a cron job monitoring the asterisk process has detected that Asterisk process was no longer running and restarted Asterisk. So Asterisk then restarted and sent 481 because it was now a newly reloaded instance of Asterisk that had no prior knowledge of the previous instance that had segfaulted. By: Joshua C. Colp (jcolp) 2020-12-09 11:29:25.562-0600 I understand that the 481 is a red herring. I gave it as an example of the only SIP response that I'm seeing in the attached pcap file. All the other SIP responses aren't present. Zip. Nada. Your analysis of the pcap is also incorrect in regards to call2. There are additional re-INVITEs to Asterisk before the T.38 re-INVITE from Asterisk outward, which I believe is the problem. I wanted the other SIP responses so I would have full SDP offer/answer negotiations for the SIPp scenario to precisely recreate that exchange. If you can't provide those then fine, we will attempt to guess and deduce based on the information we do have available. By: Gregory Massel (gmza) 2020-12-09 11:50:54.982-0600 I see what you're saying in terms of sip_trace_2 / call2. It seems that the iptables rule must have not been set up correctly, in which case we don't have the responses. With regard to the in-between re-INVITE, this was Asterisk trying to set up direct media from 196.216.192.4 to 196.216.192.5 (OpenSIPS proxy and RTPengine); i.e. Asterisk removed itself from the audio stream. However, in terms of sip_trace / call1, I do see the response codes. Also the arrangement is a lot simpler in that example as there was no media bypass to the proxy (because call recording was required in that scenario so Asterisk remained in the media path) and therefore no re-INVITE on call connect. By: Joshua C. Colp (jcolp) 2020-12-09 11:54:20.924-0600 What does each pcap represent? Completely separate calls? By: Joshua C. Colp (jcolp) 2020-12-09 11:55:21.515-0600 And what are the actual call scenarios? Passthrough? T.38 Gatewaying? SendFax/ReceiveFax? By: Gregory Massel (gmza) 2020-12-09 15:26:51.427-0600 The pcaps are completely separate unrelated calls. There were three calls at different times on three different asterisk instances (all running 16.15.0) that triggered the segfault. Of those, I was able to isolate two of them and, accordingly, attached the PCAPs and traces for the two calls I could find. In respect of the third, I'm struggling to isolate the exact call that triggered the segfault, hence haven't yet found the applicable PCAP. All three were communicating with another (forth) Asterisk box (running 16.11.1) which has had no issues at all. That box (196.216.192.4) is the one that initiated T.38 through in-dialog re-INVITE. The scenarios here were all T.38 passthrough. No gatewaying, nor SendFax/ReceiveFax applications, nor local channels. The bridged channel is PJSIP endpoint via Dial() application that is served by an OpenSIPS proxy (which acts as a registrar and proxies requqests to various different CPE devices). In the one example, Asterisk 16.15.0 was doing call recording and therefore bridged the media locally, while, in the second example, no recording took place and Asterisk 16.15.0 removed itself from the media path and directed media directly from the Asterisk 16.11.1 box to the OpenSIPS box. Since Asterisk 16.15.0 crashed in both scenarios, it is unlikely that the direct media / media bypass has any relevance. Similarly, I don't believe the proxy to be an issue as there is no issue when the Asterisk boxes are running on 16.12.0. The downgrading from 16.15.0 to 16.12.0 was approximately 24 hours ago now and all three boxes have been 100% stable since (as they were prior to upgrading to 16.15.0). By: Gregory Massel (gmza) 2020-12-09 16:36:41.564-0600 I have just tried to replicate this with a test scenario. When I use a fax machine connected to a Cisco ATA and Asterisk 16.15.0 (via the same setup with same proxy, etc.), I cannot replicate the problem. So it's not generic to every fax relay scenario. I went back to the original example and tried to locate the trace of the other leg of the call. I don't have from Asterisk to the proxy, but do have from the proxy towards the phone, and here is where it gets very strange: INVITE sip:seesa2047@102.131.29.212:16314;transport=udp SIP/2.0 Via: SIP/2.0/UDP 196.216.192.5:5060;branch=z9hG4bK6747.3a7d53f1.0 From: <sip:0128100926@vpbx2.switchtel.co.za:5060;user=phone>;tag=58493a45-b66d-49a3-8073-d581c299d123 To: "PTA-Darylize Buys" <sip:seesa2047@vpbx2.switchtel.co.za:5060>;tag=29169ea938 Contact: <sip:196.216.192.5;did=68e.5bf47d44> Call-ID: 4a06bebaf9de53be CSeq: 21413 INVITE Allow: OPTIONS, REGISTER, SUBSCRIBE, NOTIFY, PUBLISH, INVITE, ACK, BYE, CANCEL, UPDATE, PRACK, MESSAGE, REFER Supported: 100rel, timer, replaces, norefersub, histinfo Session-Expires: 1800 Min-SE: 90 Max-Forwards: 69 User-Agent: SwitchTelecom Content-Type: application/sdp Content-Length: 436 v=0 o=- 0 4 IN IP4 196.216.192.5 s=SwitchTelecom c=IN IP4 196.216.192.5 t=0 0 m=audio 14430 RTP/AVP 18 101 a=maxptime:230 a=rtpmap:18 G729/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:18 annexb=no a=fmtp:101 0-16 a=sendrecv a=rtcp:14431 a=ptime:20 m=image 17424 udptl t38 a=T38FaxVersion:0 a=T38MaxBitRate:14400 a=T38FaxRateManagement:transferredTCF a=T38FaxMaxDatagram:0 a=T38FaxUdpEC:t38UDPRedundancy a=sendrecv Note that the SDP above is very different from what was sent by the Asterisk 16.11.1 box to the Asterisk 16.15.0 box (as in the PCAP). In that case, we had SDP like: v=0 o=- 1504033814 1504033817 IN IP4 196.216.192.4 s=SwitchTelecom c=IN IP4 196.216.192.4 t=0 0 m=image 18033 udptl t38 a=T38FaxVersion:0 a=T38MaxBitRate:14400 a=T38FaxRateManagement:transferredTCF a=T38FaxMaxDatagram:494 a=T38FaxUdpEC:t38UDPRedundancy Unfortunately, without that middle leg, it's impossible for me to be certain that the proxy didn't mess up the SDP as part of its NAT traversal handling (RTPengine). That being said, I have only had this problem after the Asterisk upgrade and it hasn't recurred since downgrading to 16.12.0. Is it possible that some change might have resulted in newer Asterisk versions generating the above (corrupted) SDP with both audio and T38 in the same SDP? That is where I feel this is most likely to be, however, I cannot explain why, if that is the case, it is NOT corrupting the SDP when I try and replicate the scenario with a T.38 fax call. In my test environment, the SDP comes through perfectly (with just the T.38 part, not audio part). By: Gregory Massel (gmza) 2020-12-09 17:02:18.562-0600 Attached is the matching 2nd leg of the call for which I previously provided call2.pcap. This one does capture comms to/from the proxy to/from both the CPE and Asterisk. From this, we can prove that the proxy didn't corrupt the SDP (by including both audio and T.38) but rather the Asterisk 16.15.0 box did. By: Kevin Harwell (kharwell) 2021-01-19 10:41:33.455-0600 [~gmza] Do you still have at least one of the core dumps available still? If so would you be able, and willing to send the full core file, and libs to us? If so please see below. Required disclaimer, and how to email: {panel:title=Private Submission of Information Disclaimer}You have indicated that you wish to submit unredacted information privately. It is not recommended to do this as it will substantially restrict the number of individuals who can help with your issue, as submitted information is only available to Sangoma. Note that submission of such information does not change the priorization of this issue. If you still wish to proceed you may do so by sending it to asteriskteam@digium.com with the issue number in the subject. For large files please send a link where they can then be downloaded. By sending this information you agree to the Website Terms of Use available on the Sangoma website at https://www.sangoma.com/legal/. Any exchange of private information between you and an Asterisk community member outside of the Asterisk JIRA is not subject to the Website Terms of Use and should be privately discussed between yourself and the Asterisk community member.{panel} You can execute the {{ast_coredumper}} script to package up the necessary files. For example: {noformat} $ /var/lib/asterisk/scripts/ast_coredumper --tarball-coredumps --no-default-search <path_to_coredump> {noformat} By: Gregory Massel (gmza) 2021-01-21 11:46:15.053-0600 I did preserve the core files, however, didn't think to preserve the original asterisk binary and library against which the core files were dumped. Lesson learned; I will run ast_coredumper as you've suggested in the future. The problem is that, if I run ast_coredumper now, it will reference the binary and library for version 16.12.0 (which I'm not running), not 16.15.0 (which had the issue). If you like, I can recompile 16.15.0 with the same settings as I originally did to try and recreate the binaries and libraries, however, I'm not sure if external factors (e.g. OS updates of various generic libraries like libc) will result in the recompiled binary being indentical to what it was before. Should I send the core dumps or are they useless without the applicable binaries? By: Kevin Harwell (kharwell) 2021-01-21 14:39:27.768-0600 As you have surmised the core file is useless pretty much without the other binaries. However, it might work to recompile with same settings. Not quite sure if it will work, but worth trying out. Thanks! By: Gregory Massel (gmza) 2021-01-21 16:04:09.885-0600 I'm going to upgrade to 16.16.0 and try my best to replicate the segfault. If I am able to replicate it, then I'll make a new tarball. If unable to replicate it, I'll then fall back to recompiling 16.15.0, however, I think it probably makes sense - even if disruptive to me - for us to get another clean coredump with binaries and, at the same time, to do so against the latest version (16.16.0). By: Kevin Harwell (kharwell) 2021-01-21 16:15:39.610-0600 Since you are attempting to replicate can you also enable (in _logger.conf_), and bump the debug level in Asterisk up to say "3", and enable SIP tracing in the log file [1]. For example: {noformat} *CLI> core set debug 3 *CLI> pjsip set logger on {noformat} There is a fair amount of debugging around handling of the session that could also prove helpful. [1] https://wiki.asterisk.org/wiki/display/AST/Collecting+Debug+Information By: Gregory Massel (gmza) 2021-01-21 16:46:23.096-0600 I've managed to replicate the problem with absolute repeatability, so I've generated a new core dump tarball which can be downloaded from: http://vpbx4.switchtel.co.za/coredump/core-16.15.0.tar.gz I will remove this after a few days so ask that you please save a local copy. It is too large to attach to this ticket. I'm also attaching two PCAP's to this ticket: 1. The PCAP from the VoIP Phone the initiates the call (169.0.168.32) to the OpenSIPS proxy (196.216.192.5) to the Asterisk 16.15.0 switchboard that SEGFAULTs (196.216.192.22); and 2. The PCAP from the Asterisk 16.15.0 switchboard that SEGFAULTs (196.216.192.22) to an upstream Asterisk 16.11.1 box that switches the call onward to the phone network. Below is the console output (verbose 9, debug 9) from the Asterisk box 196.216.192.22 as it segfaults: -- Called PJSIP/0875500000/sip:0114551322@196.216.192.4:5060 -- PJSIP/0875500000-00000001 is making progress passing it to PJSIP/swt11-00000000 > 0x7fdd3c053770 -- Strict RTP learning after remote address set to: 196.216.192.4:13388 > 0x7fdd3c050bd0 -- Strict RTP learning after remote address set to: 196.216.192.5:27798 -- PJSIP/0875500000-00000001 is making progress passing it to PJSIP/swt11-00000000 > 0x7fdd3c053770 -- Strict RTP switching to RTP target address 196.216.192.4:13388 as source > 0x7fdd3c050bd0 -- Strict RTP switching to RTP target address 196.216.192.5:27798 as source -- PJSIP/0875500000-00000001 is making progress passing it to PJSIP/swt11-00000000 -- PJSIP/0875500000-00000001 is making progress passing it to PJSIP/swt11-00000000 -- PJSIP/0875500000-00000001 answered PJSIP/swt11-00000000 -- PJSIP/0875500000-00000001 Internal Gosub(swvpbx-sub-record-call-outbound,s,1(swt/swt-1611267447.0-0)) start -- Executing [s@swvpbx-sub-record-call-outbound:1] MixMonitor("PJSIP/0875500000-00000001", "/data/vpbx4.switchtel.co.za/recordings/swt/swt-1611267447.0-0.gsm,ai(__MIXMONID)") in new stack -- Executing [s@swvpbx-sub-record-call-outbound:2] Return("PJSIP/0875500000-00000001", "") in new stack == Begin MixMonitor Recording PJSIP/0875500000-00000001 == Spawn extension (swvpbx-incoming, , 1) exited non-zero on 'PJSIP/0875500000-00000001' -- PJSIP/0875500000-00000001 Internal Gosub(swvpbx-sub-record-call-outbound,s,1(swt/swt-1611267447.0-0)) complete GOSUB_RETVAL= -- Channel PJSIP/0875500000-00000001 joined 'simple_bridge' basic-bridge <3ec08c48-d1dd-4bc4-a759-7017faa6573e> -- Channel PJSIP/swt11-00000000 joined 'simple_bridge' basic-bridge <3ec08c48-d1dd-4bc4-a759-7017faa6573e> [Jan 22 00:17:28] WARNING[32523]: udptl.c:836 calculate_local_max_datagram: UDPTL (no tag): Cannot calculate local_max_datagram before local_max_ifp has been set. I will shortly repeat the same exercise using Asterisk 16.16.0 to see if the behavior is the same. By: Gregory Massel (gmza) 2021-01-21 16:46:52.940-0600 These are the PCAPs per previous comment. By: Gregory Massel (gmza) 2021-01-21 16:53:41.037-0600 The exact same thing happens on Asterisk 16.16.0. Console output is identical. Another core dump tarball can be found at: http://vpbx4.switchtel.co.za/coredump/core-16.16.0.tar.gz By: Gregory Massel (gmza) 2021-01-21 17:10:46.824-0600 Attached file verbose-log contains a verbose debug log with pjsip tracing on. By: Kevin Harwell (kharwell) 2021-01-21 17:44:38.340-0600 I've downloaded both tarballs, so feel free to delete. Thanks much for replicating, and the added information! By: Kevin Harwell (kharwell) 2021-02-01 16:38:28.795-0600 [~gmza] I've attached a patch, [^ASTERISK-29203.diff] against Asterisk 16. If you can please apply and test it and let me know if it fixes your issue. Thanks! By: Gregory Massel (gmza) 2021-02-01 19:32:17.422-0600 I've tested against 16.16 and it works perfectly! I first replicated the scenario and crashed it, then applied the patch and repeated the test, and it worked fine, both in the scenario of no T.38 changeover (when I initiated from a phone) as well as in the scenario with T.38 changeover (when I initiated from a Fax + ATA). Thank you so much for the effort that has gone into this; it is greatly appreciated! By: Kevin Harwell (kharwell) 2021-02-02 09:27:43.746-0600 Excellent news! Thanks for testing out the patch. I'm glad it was able to fix your issue. By: Kevin Harwell (kharwell) 2021-02-04 12:50:12.931-0600 Attaching security document description [^AST-2021-002.pdf] for review. By: Gregory Massel (gmza) 2021-02-05 08:00:28.072-0600 It was unclear from the previous comment if 'for review' was directed to me (as reporter) or to the Asterisk team. FWIW, the document looks good. By: Kevin Harwell (kharwell) 2021-02-05 09:48:27.053-0600 [~gmza] oh ya apologies for the ambiguity. We'll usually attach the document to the issue for the reporter to review, but also any other community members that might have access just to make sure the wording sounds good, and there is nothing else that needs to be clarified or added. Thanks for reviewing it! By: Friendly Automation (friendly-automation) 2021-02-18 10:40:09.664-0600 Change 15473 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15473|https://gerrit.asterisk.org/c/asterisk/+/15473] By: Friendly Automation (friendly-automation) 2021-02-18 10:40:12.790-0600 Change 15472 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15472|https://gerrit.asterisk.org/c/asterisk/+/15472] By: Friendly Automation (friendly-automation) 2021-02-18 10:40:15.151-0600 Change 15482 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15482|https://gerrit.asterisk.org/c/asterisk/+/15482] By: Friendly Automation (friendly-automation) 2021-02-18 10:40:17.661-0600 Change 15474 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15474|https://gerrit.asterisk.org/c/asterisk/+/15474] By: Friendly Automation (friendly-automation) 2021-02-18 10:40:22.014-0600 Change 15471 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15471|https://gerrit.asterisk.org/c/asterisk/+/15471] By: Friendly Automation (friendly-automation) 2021-02-18 10:40:24.738-0600 Change 15484 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15484|https://gerrit.asterisk.org/c/asterisk/+/15484] By: Friendly Automation (friendly-automation) 2021-02-18 10:40:27.354-0600 Change 15481 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15481|https://gerrit.asterisk.org/c/asterisk/+/15481] By: Friendly Automation (friendly-automation) 2021-02-18 10:41:01.378-0600 Change 15475 merged by George Joseph: AST-2021-002: Remote crash possible when negotiating T.38 [https://gerrit.asterisk.org/c/asterisk/+/15475|https://gerrit.asterisk.org/c/asterisk/+/15475] |