[Home]

Summary:ASTERISK-09952: segfault on reload
Reporter:Clod Patry (junky)Labels:
Date Opened:2007-07-24 19:58:31Date Closed:2007-07-26 13:32:36
Priority:MinorRegression?No
Status:Closed/CompleteComponents:CDR/cdr_pgsql
Versions:Frequency of
Occurrence
Related
Issues:
Environment:Attachments:( 0) 10295.patch
Description:After an upgrade to 1.4.9
I've that weird segfault.

****** ADDITIONAL INFORMATION ******

(gdb) bt
#0  0x00002aaaab5493f1 in SSL_shutdown () from /usr/lib/libssl.so.0.9.8
#1  0x00002aaaab0fae25 in ?? () from /usr/lib/libpq.so.5
#2  0x00002aaaab0eda1a in ?? () from /usr/lib/libpq.so.5
#3  0x00002aaaab0edcfe in PQfinish () from /usr/lib/libpq.so.5
#4  0x00002aaabd324014 in my_unload_module () at cdr_pgsql.c:188
ASTERISK-1  0x00002aaabd3247d5 in reload () at cdr_pgsql.c:319
ASTERISK-2  0x0000000000471df8 in ast_module_reload (name=0x0) at loader.c:590
ASTERISK-3  0x000000000044bb23 in handle_reload_deprecated (fd=34, argc=1, argv=0x407bab90) at cli.c:182
ASTERISK-4  0x00000000004524f6 in ast_cli_command (fd=34, s=0x407badf0 "reload") at cli.c:1979
ASTERISK-5  0x000000000042a673 in netconsole (vconsole=0x758800) at asterisk.c:932
ASTERISK-6 0x00000000004c8808 in dummy_start (data=0x7c2750) at utils.c:545
ASTERISK-7 0x00002abda2b872a5 in start_thread () from /lib/libpthread.so.0
ASTERISK-8 0x00002abda3a7261d in clone () from /lib/libc.so.6
ASTERISK-9 0x0000000000000000 in ?? ()
(gdb) bt full
#0  0x00002aaaab5493f1 in SSL_shutdown () from /usr/lib/libssl.so.0.9.8
No symbol table info available.
#1  0x00002aaaab0fae25 in ?? () from /usr/lib/libpq.so.5
No symbol table info available.
#2  0x00002aaaab0eda1a in ?? () from /usr/lib/libpq.so.5
No symbol table info available.
#3  0x00002aaaab0edcfe in PQfinish () from /usr/lib/libpq.so.5
No symbol table info available.
#4  0x00002aaabd324014 in my_unload_module () at cdr_pgsql.c:188
No locals.
ASTERISK-1  0x00002aaabd3247d5 in reload () at cdr_pgsql.c:319
       res = 10922
ASTERISK-2  0x0000000000471df8 in ast_module_reload (name=0x0) at loader.c:590
       info = (const struct ast_module_info *) 0x2aaabd5263c0
       cur = (struct ast_module *) 0x810cd0
       res = 2
       i = 7
ASTERISK-3  0x000000000044bb23 in handle_reload_deprecated (fd=34, argc=1, argv=0x407bab90) at cli.c:182
       x = 7612080
       res = 0
ASTERISK-4  0x00000000004524f6 in ast_cli_command (fd=34, s=0x407badf0 "reload") at cli.c:1979
       argv = {0x807ee0 "reload", 0x0, 0x8348d9 "", 0x8348df "", 0x8348e7 "", 0x0, 0x407bafef "", 0x0, 0x0, 0x0, 0x0, 0x0, 0x407bbd80 "",
 0x2abda3a6b855 "1?L9?v?H\213\0055'(", 0x78756e0000 <Address 0x78756e0000 out of bounds>, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
 0x290079ffffffff <Address 0x290079ffffffff out of bounds>, 0x0, 0x0, 0x2abda3ced460 "", 0x0, 0x0, 0x0, 0x0,
 0x30322e362e320000 <Address 0x30322e362e320000 out of bounds>, 0x656e65672d36312d <Address 0x656e65672d36312d out of bounds>,
 0x636972 <Address 0x636972 out of bounds>, 0x0, 0x0, 0x0, 0x0, 0x407bb040 "\020?{@", 0x0, 0x407bb940 "@?{@", 0x3 <Address 0x3 out of bounds>,
 0x2abda39ef603 "H\201??", 0x3000000030 <Address 0x3000000030 out of bounds>, 0x407badc0 "@?{@", 0x407bad00 "", 0x0,
 0x5f36387800000000 <Address 0x5f36387800000000 out of bounds>, 0x3436 <Address 0x3436 out of bounds>, 0x0, 0x407baff0 "troy",
 0x6096 <Address 0x6096 out of bounds>, 0x500d2c "1.4.9", 0x0, 0x0, 0x6f6e280000000000 <Address 0x6f6e280000000000 out of bounds>,
 0x29656e <Address 0x29656e out of bounds>, 0x0, 0x2abda2b8df69 "H\213D$\bH\203?(H=\001???s\001?H\213\r\036\220 ", 0x0, 0x12 <Address 0x12 out of bounds>,
 0x407badf0 "reload", 0x12 <Address 0x12 out of bounds>, 0x0, 0x2abda2b8dfe9 "H\213D$\bH\203?(H=\001???s\001?H\213\r\236\217 "}
       e = (struct ast_cli_entry *) 0x744020
       x = 1
       dup = 0x807ee0 "reload"
       tws = 0
ASTERISK-5  0x000000000042a673 in netconsole (vconsole=0x758800) at asterisk.c:932
       con = (struct console *) 0x758800
       hostname = "troy", '\0' <repeats 59 times>
       tmp = "reload\000\000 debug atleast 0\000\000\000\000n\n\000\000??{@\000\000\000\000\000?{@", '\0' <repeats 12 times>, "\b?{@", '\0' <repeats 20 times>, "\020?{@", '\0' <repeats 36 times>, "`???*\000\000@?{@", '\0' <repeats 12 times>, "@?{@\000\000\000\000\003\000\000\000\000\000\000\000\000\000\017@\000\000\000\000lb???*\000\000@?{@\000\000\000\000\003\000\000\000\000\000\000\000\000\000\017@\000\000\000\000\236:??*", '\0' <repeats 18 times>, "@?{@\000\000\000\000\207=??*", '\0' <repeats 18 times>...
       res = 7
       fds = {{fd = 34, events = 1, revents = 1}, {fd = 35, events = 1, revents = 0}}
       __PRETTY_FUNCTION__ = "netconsole"
ASTERISK-6 0x00000000004c8808 in dummy_start (data=0x7c2750) at utils.c:545
       __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf = {0, -71810155676800290, 0, 1081850176, 3, 0, -71810155676800098, -71810154602341375},
     __mask_was_saved = 0}}, __pad = {0x407bb1b0, 0x0, 0x7b56c0, 0x0}}
       __cancel_routine = (void (*)(void *)) 0x428a63 <ast_unregister_thread>
       __cancel_arg = (void *) 0x407bb940
       not_first_call = 0
       ret = (void *) 0x2abda3cea888
---Type <return> to continue, or q <return> to quit---
       a = {start_routine = 0x42a4b3 <netconsole>, data = 0x758800,
 name = 0x807e90 "netconsole", ' ' <repeats 11 times>, "started at [  997] asterisk.c listener()"}
ASTERISK-7 0x00002abda2b872a5 in start_thread () from /lib/libpthread.so.0
No symbol table info available.
ASTERISK-8 0x00002abda3a7261d in clone () from /lib/libc.so.6
No symbol table info available.
ASTERISK-9 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb)





Comments:By: Clod Patry (junky) 2007-07-24 20:00:31

this is how, you can crash it, all the time:
   -- Executing [h@default:1] NoOp("SIP/10-008164d0", "vmstatus= ") in new stack
   -- Executing [h@default:2] GotoIf("SIP/10-008164d0", "0?h|100") in new stack
   -- Executing [h@default:3] NoOp("SIP/10-008164d0", "I hangup since VMSTATUS isnt SUCCESS") in new stack
   -- Executing [h@default:4] Hangup("SIP/10-008164d0", "") in new stack
 == Spawn extension (default, h, 4) exited non-zero on 'SIP/10-008164d0'
[Jul 24 21:12:13] ERROR[29973]: cdr_pgsql.c:90 pgsql_log: cdr_pgsql: Unable to connect to database server localhost.  Calls will not be logged!
[Jul 24 21:12:13] ERROR[29973]: cdr_pgsql.c:91 pgsql_log: cdr_pgsql: Reason: ?�?
troy*CLI> reload
troy*CLI>
Disconnected from Asterisk server


So apparently, each time psql cant connect for an insert, and a relaod, and kaboom.


If i put cdr_pgsql.c:188 in comment, there's no more segfault.


By: Mark Michelson (mmichelson) 2007-07-25 09:22:13

This area isn't exactly my specialty, but it looks like two consecutive calls to PQfinish() are made on the same pointer, meaning that the same chunk of memory is being freed twice.

I'll write up a simple patch to try and post in a few minutes.

Edit: Patch is uploaded as 10295.patch. It explicitly sets the connection to NULL after calls to PQfinish() so that unloading the module won't cause a crash.


By: Prashant Jois (prashant_jois) 2007-07-26 11:46:21

Yes, the problem is exactly as you described it putnopvut.  Your patch should take care of that.  Sorry I missed that.

(btw, you don't need to check for NULL when calling PQfinish.  PQfinish will safely accept a NULL pointer. No biggie either way though.)
By: Prashant Jois (prashant_jois) 2007-07-26 11:50:40

I just noticed another problem:

lines 88-92:

  pgerror = PQerrorMessage(conn);
  PQfinish(conn);
  ast_log(LOG_ERROR, "cdr_pgsql: Unable to connect to database server %s.  Calls will not be logged!\n", pghostname);
  ast_log(LOG_ERROR, "cdr_pgsql: Reason: %s\n", pgerror);

---------------

PQfinish should come after line 92, because pgerror is a pointer to one of the fields in conn:


  pgerror = PQerrorMessage(conn);
  ast_log(LOG_ERROR, "cdr_pgsql: Unable to connect to database server %s.  Calls will not be logged!\n", pghostname);
  ast_log(LOG_ERROR, "cdr_pgsql: Reason: %s\n", pgerror);
  PQfinish(conn);

---------------

The same goes for lines 148-152.  It should be:

  pgerror = PQerrorMessage(conn);
  ast_log(LOG_ERROR, "cdr_pgsql: Unable to reconnect to database server %s. Calls will not be logged!\n", pghostname);
  ast_log(LOG_ERROR, "cdr_pgsql: Reason: %s\n", pgerror);
  PQfinish(conn);

(I'm submitting this one as a new bug, with patch).


By: Prashant Jois (prashant_jois) 2007-07-26 12:16:55

see 0010310
By: Mark Michelson (mmichelson) 2007-07-26 13:32:35

This is fixed for 1.4 in svn revision 77318 and for trunk in svn revision 77319.