Summary:ASTERISK-29981: res_calendar: Asterisk crashes when starting, and will not run
Reporter:N A (InterLinked)Labels:
Date Opened:2022-03-22 10:30:54Date Closed:2022-06-06 16:50:00
Versions:18.9.0 Frequency of
Environment:Attachments:( 0) full.txt
Description:Asterisk has been crashing every 5 minutes for the last 3 hours. I have a script that starts it up every 5 minutes, except it crashes when started. Effectively, Asterisk will no longer run.

Backtrace when starting:

[2022-03-22 15:26:24] WARNING[29884]: sched.c:497 sched_settime: Bug likely: Negative time interval -55584000 (interpreted as 4239383296 ms) requested!
[2022-03-22 15:26:24] ERROR[29884]: sched.c:500 sched_settime: FRACK!, Failed assertion 0 (0)
[2022-03-22 15:26:25] ERROR[29884]:   Got 14 backtrace records
# 0: asterisk __ast_assert_failed()
# 1: asterisk <unknown>()
# 2: asterisk <unknown>()
# 3: asterisk ast_sched_add_variable()
# 4: [0x7f2df0a81243] res_calendar.so res_calendar.c:1002 schedule_calendar_event()
# 5: [0x7f2df0a814a1] res_calendar.so res_calendar.c:1050 add_new_event_cb()
# 6: asterisk <unknown>()
# 7: asterisk __ao2_callback()
# 8: [0x7f2df0a8153c] res_calendar.so res_calendar.c:1061 ast_calendar_merge_events()
# 9: [0x7f2df02cec26] res_calendar_icalendar.so res_calendar_icalendar.c:383 icalendar_update_events()
#10: [0x7f2df02cf769] res_calendar_icalendar.so res_calendar_icalendar.c:483 ical_load_calendar()
#11: asterisk <unknown>()
#12: [0x7f2e02b38fa3] libpthread.so.0 pthread_create.c:487 start_thread()
#13: [0x7f2e025cc4cf] libc.so.6 clone.S:97 clone()

Backtrace from ast_coredumper:

This is the first time I have had an issue where Asterisk itself does not start. res_calendar causes a crash as soon as it gets loaded.

If I add noload = res_calendar to modules.conf, Asterisk starts, but now production is impaired and operating significantly degraded.
Comments:By: Asterisk Team (asteriskteam) 2022-03-22 10:30:55.011-0500

Thanks for creating a report! The issue has entered the triage process. That means the issue will wait in this status until a Bug Marshal has an opportunity to review the issue. Once the issue has been reviewed you will receive comments regarding the next steps towards resolution. Please note that log messages and other files should not be sent to the Sangoma Asterisk Team unless explicitly asked for. All files should be placed on this issue in a sanitized fashion as needed.

A good first step is for you to review the [Asterisk Issue Guidelines|https://wiki.asterisk.org/wiki/display/AST/Asterisk+Issue+Guidelines] if you haven't already. The guidelines detail what is expected from an Asterisk issue report.

Then, if you are submitting a patch, please review the [Patch Contribution Process|https://wiki.asterisk.org/wiki/display/AST/Patch+Contribution+Process].

Please note that once your issue enters an open state it has been accepted. As Asterisk is an open source project there is no guarantee or timeframe on when your issue will be looked into. If you need expedient resolution you will need to find and pay a suitable developer. Asking for an update on your issue will not yield any progress on it and will not result in a response. All updates are posted to the issue when they occur.

Please note that by submitting data, code, or documentation to Sangoma through JIRA, you accept the Terms of Use present at [https://www.asterisk.org/terms-of-use/|https://www.asterisk.org/terms-of-use/].

By: N A (InterLinked) 2022-03-22 10:31:34.492-0500

Backtrace full.txt is attached.

By: N A (InterLinked) 2022-03-23 05:45:04.909-0500

...And, as of today, there no longer seem to be any issues. So for some reason it was constantly crashing yesterday, but today, doesn't appear to be an issue.

I suspect on the basis of the issue, for some reason it was reading in times in the past and that was causing an internal assertion to fail here: https://github.com/asterisk/asterisk/blob/master/res/res_calendar.c#L1002

I recompiled with DO_CRASH recently so this issue has likely happened before, but not caused a crash until now.

By: Marian Piater (maikyp) 2022-05-30 08:54:24.730-0500

Probably similar crash (I don't have coredump, because server was restarted). I am waiting for another crash and coredump.

Compiled Asterisk 16.24.0.
Checked Asterisk changelog and seems there are no changes in calendar resource in newer releases.


[May 27 09:44:38] ERROR[19772] res_calendar.c: FRACK!, Failed assertion bad magic number 0x0 for object 0x7fa6f4475350 (0)
[May 27 09:44:38] ERROR[19772] : Got 7 backtrace records
# 0: /usr/sbin/asterisk(__ao2_ref+0x9e) [0x55f57ab04aee]
# 1: /usr/lib/asterisk/modules/res_calendar.so(+0x576d) [0x7fa6c363f76d]
# 2: /usr/sbin/asterisk(ast_sched_runq+0xfa) [0x55f57ac25a2a]
# 3: /usr/lib/asterisk/modules/res_calendar.so(+0x3774) [0x7fa6c363d774]
# 4: /usr/sbin/asterisk(+0x1ce84c) [0x55f57ac6684c]
# 5: /lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4) [0x7fa71b6364a4]
# 6: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fa71a241d0f]

By: N A (InterLinked) 2022-05-30 08:57:43.255-0500

Please try the patch attached here and see if it resolves your issue: https://gerrit.asterisk.org/c/asterisk/+/18250

This has not yet been merged into an Asterisk release.

By: Friendly Automation (friendly-automation) 2022-06-06 16:50:00.877-0500

Change 18250 merged by Friendly Automation:
res_calendar: Prevent assertion if event ends in past.


By: Friendly Automation (friendly-automation) 2022-06-06 16:55:29.384-0500

Change 18594 merged by Kevin Harwell:
res_calendar: Prevent assertion if event ends in past.


By: Friendly Automation (friendly-automation) 2022-06-06 17:38:30.605-0500

Change 18595 merged by Friendly Automation:
res_calendar: Prevent assertion if event ends in past.


By: Friendly Automation (friendly-automation) 2022-06-06 17:45:02.267-0500

Change 18596 merged by Kevin Harwell:
res_calendar: Prevent assertion if event ends in past.


By: Marian Piater (maikyp) 2022-06-07 02:01:52.103-0500

Thank you. Missed your reply, I'll try to apply the patch.