I'm sorry, I have no answers, just numbers at the moment. If I start up Asterisk, and make 2 calls, from one sip phone to another, thru asterisk, with both DEBUG_THREADS and MTX_PROF set in the menuselect stuff in my mtxprof branch (team/murf/mtxprof)... My mutex profiling reveals a few top busy waiters: number scale events busy totalclks avg lock at: (who had the lock [recursive cnt]) 134: [ 0] 40 1 202429 5060 chan_mgcp.c__do_monitor__3447__&monlock (chan_mgcp.c:3507:restart_monitor:&monlock[1]) 16: [ 0] 1239 3 3024368 2440 astobj2.c__ao2_lock__154__&p->priv_data.lock (astobj2.c:154:ao2_lock:&p->priv_data.lock[1]) 251: [ 0] 870 3 3525943 4052 channel.c__ast_waitfor_nandfds__1878__&c[x]->lock_dont_use (chan_sip.c:18893:handle_request_do:&c[x]->lock_dont_use[2]) 256: [ 0] 430 4 251589 585 channel.c____ast_read__2391__&chan->lock_dont_use (chan_sip.c:18893:handle_request_do:&chan->lock_dont_use[1]) 257: [ 0] 1 36 588 588 channel.c____ast_read__2397__&chan->lock_dont_use (chan_sip.c:18893:handle_request_do:&chan->lock_dont_use[1]) The above is sorted by the "busy" column. Basically, if you call a mutex_lock() and it's busy at the time of the call, then the busy number is incremented. The avg column is the average number of cpu ticks to get the lock. The events column records the number of times that this particular lock was called. In parens, at the end of the line, is who had the lock when it was busy. The 'lock at' column has names in this format: filename_funcname_lineno_lock We can toss out the chan_mgcp locks, as we are not concerned about them. On this particular computer, The cpu runs at 2793 MHz, so to get microsec, divide cpu ticks by 2793. So, the avg wait times for astobj2's lock is pretty low, about a microsecond. Also, it got called 1239 times, and only had to wait 3 times. The __ast_waitfor_nandfs has an avg wait time of around 2 microseconds, and only had to wait 3 times, but those 3 times might have been fairly long waits. judging that it dragged the avg up to twice the astobj2 call. All these seem to be waiting on a handle_request_do lock. handle_request_do must not hold it long, the avg 588 msec seems pretty short for a routine that had to wait almost every time it was called. Speaking of handle_request_do, here are the figures on it's locking: 253: [ 0] 10 0 167412 16741 chan_sip.c__handle_request_do__18893__&p->owner->lock_dont_use 254: [ 0] 10 0 9267878 926787 UNLOCK:chan_sip.c__handle_request_do__18933__&p->owner->lock_dont_use In these entries, the first shows that handle_request_do was called 10 times, never had to wait for its lock, and took an avg of 16741 clicks to get it. The second entry records the release of that lock, the avg time being how many cpu ticks the lock was held. 926,787 seems a bit long of a time to hold a channel lock... as long as it doesn't hold things up... that's 331 microseconds, or .331 milliseconds. There is just one other mention of handle_request_do; twice (once for each call?) the ast_queue_hangup released that lock instead of the handle_request_do func itself: 285: [ 0] 2 0 11759 5879 UNLOCK:channel.c__ast_queue_hangup__1020__&chan->lock_dont_use (chan_sip.c:18893:handle_request_do:&chan->lock_dont_use[1]) So far, I don't see anything that would cause me to jump up and down and say, hey, this is what is slowing down chan_sip. WITH VARIOUS CDR BACKENDS ACTIVE: So, let's look at the highest lock-grant/lock-hold times, and see who's near the top of the list: (the list has a minimum avg grant/hold time of 100k, or around 35 usec on this machine): 207: [ 10] 14 0 51254 3661 channel_find_locked 172: [ 0] 1 0 104906 104906 UNLOCK:db.c__ast_db_deltree__134__&dblock 92: [ 0] 3 0 332567 110855 UNLOCK:db.c__ast_db_gettree__528__&dblock 144: [ 0] 77 0 13410608 174163 UNLOCK:chan_sip.c__do_monitor__19316__&monlock 114: [ 0] 1 0 215103 215103 UNLOCK:chan_sip.c__sip_send_all_registers__22138__&(®l)->_lock 57: [ 0] 41 0 8900337 217081 UNLOCK:devicestate.c__run_devstate_collector__694__&devstate_collector.lock 281: [ 0] 2 0 479766 239883 UNLOCK:rtp.c__ast_rtp_bridge__4344__&c1->lock_dont_use 280: [ 0] 2 0 522250 261125 UNLOCK:rtp.c__ast_rtp_bridge__4343__&c0->lock_dont_use 96: [ 0] 1 0 262468 262468 UNLOCK:chan_skinny.c__restart_monitor__6087__&monlock 124: [ 0] 1 0 270354 270354 UNLOCK:chan_phone.c__restart_monitor__1156__&monlock 50: [ 0] 122 0 34314209 281264 UNLOCK:taskprocessor.c__tps_processing_function__300__&i->taskprocessor_lock 258: [ 0] 8 0 2273545 284193 UNLOCK:channel.c__ast_queue_frame__1007__&chan->lock_dont_use (channel.c:1577:ast_softhangup:&chan->lock_dont_use[1]) 275: [ 0] 2 0 798367 399183 UNLOCK:channel.c____ast_answer__1735__&chan->lock_dont_use 178: [ 0] 1 0 413375 413375 UNLOCK:chan_dahdi.c__restart_monitor__8084__&monlock 263: [ 0] 2 0 983325 491662 UNLOCK:rtp.c__ast_rtp_early_bridge__2065__&c1->lock_dont_use 290: [ 0] 2 0 996325 498162 UNLOCK:cdr_custom.c__custom_log__132__&mf_lock 135: [ 0] 1 0 505407 505407 UNLOCK:chan_mgcp.c__restart_monitor__3528__&monlock 262: [ 0] 2 0 1033871 516935 UNLOCK:rtp.c__ast_rtp_early_bridge__2063__&c0->lock_dont_use 70: [ 0] 1 0 520905 520905 UNLOCK:res_odbc.c__odbc_obj_disconnect__768__&obj->lock 43: [ 0] 1 0 532760 532760 UNLOCK:cdr.c__do_reload__1483__&cdr_batch_lock 150: [ 0] 30 0 19729164 657638 UNLOCK:devicestate.c__do_devstate_changes__555__&(&state_changes)->lock 130: [ 0] 1 0 672326 672326 UNLOCK:chan_mgcp.c__reload_config__4266__&netlock 118: [ 0] 3 0 2062830 687610 UNLOCK:chan_sip.c__restart_monitor__19349__&monlock 217: [ 0] 39 0 28296251 725544 UNLOCK:chan_dahdi.c__do_monitor__8055__&iflock 206: [ 0] 1 0 729939 729939 UNLOCK:app_minivm.c__load_config__2400__&minivmlock 298: [ 0] 4 0 3283339 820834 UNLOCK:channel.c__ast_hangup__1664__&chan->lock_dont_use 254: [ 0] 10 0 9267878 926787 UNLOCK:chan_sip.c__handle_request_do__18933__&p->owner->lock_dont_use --- above a million cpu ticks (360 usec on this machine)--- 242: [ 0] 2 0 2154054 1077027 UNLOCK:rtp.c__ast_rtp_make_compatible__2158__&src->lock_dont_use 241: [ 0] 2 0 2199770 1099885 UNLOCK:rtp.c__ast_rtp_make_compatible__2157__&dest->lock_dont_use 10: [ 0] 1 0 1158906 1158906 UNLOCK:utils.c__test_for_thread_safety__283__&test_lock2 94: [ 0] 1 0 1211206 1211206 UNLOCK:chan_skinny.c__reload_config__6307__&netlock 106: [ 0] 1 0 1399944 1399944 UNLOCK:chan_sip.c__reload_config__21581__&netlock 196: [ 0] 1 0 1524306 1524306 UNLOCK:app_voicemail.c__load_config__10069__&(&users)->lock 246: [ 0] 2 0 3725264 1862632 UNLOCK:channel.c__ast_call__3512__&chan->lock_dont_use 180: [ 0] 1 0 1950060 1950060 UNLOCK:chan_agent.c__read_agent_config__1247__&(&agents)->lock 149: [ 0] 32 0 103508799 3234649 UNLOCK:chan_sip.c__handle_request_do__18936__&netlock 68: [ 0] 1 0 3326925 3326925 UNLOCK:res_odbc.c__odbc_obj_connect__812__&obj->lock 273: [ 0] 2 0 8128995 4064497 UNLOCK:channel.c____ast_answer__1726__&chan->lock_dont_use 166: [ 0] 1 0 4510663 4510663 UNLOCK:chan_iax2.c__reload_firmware__2127__&(&firmwares)->lock 90: [ 0] 1 0 6556305 6556305 UNLOCK:func_odbc.c__load_module__820__&(&queries)->lock 220: [ 0] 2 0 13712486 6856243 pbx_start_time 174: [ 0] 1 0 9341083 9341083 UNLOCK:chan_dahdi.c__setup_dahdi__14763__&iflock 267: [ 0] 2 0 20279878 10139939 UNLOCK:db.c__ast_db_put__159__&dblock 292: [ 0] 2 0 53385518 26692759 UNLOCK:cdr_sqlite.c__sqlite_log__158__&sqlite_lock 216: [ 0] 39 0 1557488348 39935598 UNLOCK:chan_iax2.c__sched_thread__10101__&sched_lock 33: [ 0] 2 0 145779269 72889634 UNLOCK:loader.c__load_modules__951__&(&module_list)->lock I highlighted in orange the ones most likely involved with chan_sip; the pbx_start time is alarmingly high. The rest will take some research to determine if they are a problem or no big deal. The above is two calls, no loading. Now, let's see how things stand in a high load environment. Using sipp, rapid fire short calls. Pure call setup and teardown. Now, for 316 calls placed at a rate from 10 per sec to 90 per sec for a fairly short span of seconds... 249: [ 0] 0 3476 0 0 channel.c____ast_read__2407__&chan->lock_dont_use (chan_sip.c:18893:handle_request_do:&chan->lock_dont_use[1]) 174: [ 0] 1 0 99088 99088 UNLOCK:db.c__ast_db_deltree__134__&dblock 94: [ 0] 3 0 332661 110887 UNLOCK:db.c__ast_db_gettree__528__&dblock 239: [ 0] 1893 0 240051250 126809 UNLOCK:channel.c__ast_write__3224__&chan->lock_dont_use 41: [ 0] 1 0 159810 159810 UNLOCK:cdr.c__do_reload__1483__&cdr_batch_lock 70: [ 0] 1 0 172252 172252 UNLOCK:res_odbc.c__odbc_obj_disconnect__768__&obj->lock 116: [ 0] 1 0 194008 194008 UNLOCK:chan_sip.c__sip_send_all_registers__22138__&(®l)->_lock 98: [ 0] 1 0 241584 241584 UNLOCK:chan_skinny.c__restart_monitor__6087__&monlock 126: [ 0] 1 0 256295 256295 UNLOCK:chan_phone.c__restart_monitor__1156__&monlock 216: [ 10] 1430 0 41311969 28889 channel_find_locked 48: [ 0] 4301 0 1254821914 291751 UNLOCK:taskprocessor.c__tps_processing_function__300__&i->taskprocessor_lock 137: [ 0] 1 0 296803 296803 UNLOCK:chan_mgcp.c__restart_monitor__3528__&monlock 55: [ 0] 1434 0 490421060 341995 UNLOCK:devicestate.c__run_devstate_collector__694__&devstate_collector.lock 260: [ 0] 316 0 110274933 348971 UNLOCK:channel.c__ast_hangup__1675__&chan->lock_dont_use 180: [ 0] 1 0 405667 405667 UNLOCK:chan_dahdi.c__restart_monitor__8084__&monlock 167: [ 0] 1323 0 576553388 435792 UNLOCK:devicestate.c__do_devstate_changes__555__&(&state_changes)->lock 231: [ 0] 316 0 149426831 472869 UNLOCK:channel.c____ast_answer__1754__&chan->lock_dont_use 243: [ 0] 632 0 361391961 571822 UNLOCK:chan_sip.c__handle_request_do__18933__&p->owner->lock_dont_use 223: [ 0] 316 0 188588475 596798 pbx_start_time 96: [ 0] 1 0 619175 619175 UNLOCK:chan_skinny.c__reload_config__6307__&netlock 182: [ 0] 1 0 650002 650002 UNLOCK:chan_agent.c__read_agent_config__1247__&(&agents)->lock 250: [ 0] 316 0 250099435 791453 UNLOCK:channel.c__ast_queue_frame__1007__&chan->lock_dont_use (chan_sip.c:18893:handle_request_do:&chan->lock_dont_use[1]) 215: [ 0] 25 0 21475619 859024 UNLOCK:chan_dahdi.c__do_monitor__8055__&iflock 212: [ 0] 1 0 983104 983104 UNLOCK:app_minivm.c__load_config__2400__&minivmlock 202: [ 0] 1 0 1472509 1472509 UNLOCK:app_voicemail.c__load_config__10069__&(&users)->lock 120: [ 0] 1 0 1568564 1568564 UNLOCK:chan_sip.c__restart_monitor__19349__&monlock 266: [ 0] 92 0 188648065 2050522 UNLOCK:cdr_custom.c__custom_log__132__&mf_lock 68: [ 0] 1 0 2499976 2499976 UNLOCK:res_odbc.c__odbc_obj_connect__812__&obj->lock 92: [ 0] 1 0 2879604 2879604 UNLOCK:func_odbc.c__load_module__820__&(&queries)->lock 176: [ 0] 1 0 3866163 3866163 UNLOCK:chan_dahdi.c__setup_dahdi__14763__&iflock 86: [ 0] 3 0 52887723 17629241 UNLOCK:db.c__ast_db_put__159__&dblock 213: [ 0] 25 0 1109035233 44361409 UNLOCK:chan_iax2.c__sched_thread__10101__&sched_lock 168: [ 0] 1 0 48314889 48314889 UNLOCK:chan_iax2.c__reload_firmware__2127__&(&firmwares)->lock 23: [ 0] 2 0 105186344 52593172 UNLOCK:loader.c__load_modules__951__&(&module_list)->lock 267: [ 0] 92 14 -1735201542 -18860886 cdr_sqlite.c__sqlite_log__98__&sqlite_lock (cdr_sqlite.c:98:sqlite_log:&sqlite_lock[1]) 268: [ 0] 92 0 -1709752067 -18584261 UNLOCK:cdr_sqlite.c__sqlite_log__158__&sqlite_lock 166: [ 0] 958 0 -931542010 -972382 UNLOCK:chan_sip.c__handle_request_do__18936__&netlock 17: [ 0] 18228 0 -346683323 -19019 UNLOCK:astobj2.c__ao2_unlock__170__&p->priv_data.lock Notes, 1. I scaled the units for measuring channel_find_locked, and set it in the proper place in the list. 2. I don't know why ast_read doesn't have time accumulated. perhaps if it is busy, I don't add in the wait time. 3. negative entries overflowed on the totals. 4. channel_find_locked takes 10 times the cpu cycles under load than it does in unloaded conditions. WITH CDR BACKENDS TURNED OFF: (2 calls, one after the other, sip phone to sip phone). LOW LOAD: 207: [ 10] 14 0 40553 2896 channel_find_locked 268: [ 0] 8 0 855940 106992 UNLOCK:channel.c__ast_indicate_data__2835__&chan->lock_dont_use 92: [ 0] 3 0 333743 111247 UNLOCK:db.c__ast_db_gettree__528__&dblock 142: [ 0] 102 0 18626129 182609 UNLOCK:chan_sip.c__do_monitor__19316__&monlock 57: [ 0] 40 0 8472205 211805 UNLOCK:devicestate.c__run_devstate_collector__694__&devstate_collector.lock 114: [ 0] 1 0 215460 215460 UNLOCK:chan_sip.c__sip_send_all_registers__22138__&(®l)->_lock 282: [ 0] 2 0 489594 244797 UNLOCK:rtp.c__ast_rtp_bridge__4344__&c1->lock_dont_use 96: [ 0] 1 0 264936 264936 UNLOCK:chan_skinny.c__restart_monitor__6087__&monlock 281: [ 0] 2 0 534713 267356 UNLOCK:rtp.c__ast_rtp_bridge__4343__&c0->lock_dont_use 124: [ 0] 1 0 268516 268516 UNLOCK:chan_phone.c__restart_monitor__1156__&monlock 50: [ 0] 119 0 34133965 286840 UNLOCK:taskprocessor.c__tps_processing_function__300__&i->taskprocessor_lock 261: [ 0] 8 0 2367048 295881 UNLOCK:channel.c__ast_queue_frame__1007__&chan->lock_dont_use (channel.c:1588:ast_softhangup:&chan->lock_dont_use[1]) 135: [ 0] 1 0 349450 349450 UNLOCK:chan_mgcp.c__restart_monitor__3528__&monlock 178: [ 0] 1 0 427192 427192 UNLOCK:chan_dahdi.c__restart_monitor__8084__&monlock 276: [ 0] 2 0 907547 453773 UNLOCK:channel.c____ast_answer__1745__&chan->lock_dont_use 266: [ 0] 2 0 990381 495190 UNLOCK:rtp.c__ast_rtp_early_bridge__2065__&c1->lock_dont_use 70: [ 0] 1 0 517041 517041 UNLOCK:res_odbc.c__odbc_obj_disconnect__768__&obj->lock 265: [ 0] 2 0 1035237 517618 UNLOCK:rtp.c__ast_rtp_early_bridge__2063__&c0->lock_dont_use 218: [ 0] 51 0 29385563 576187 UNLOCK:chan_dahdi.c__do_monitor__8055__&iflock 130: [ 0] 1 0 668482 668482 UNLOCK:chan_mgcp.c__reload_config__4266__&netlock 43: [ 0] 1 0 752640 752640 UNLOCK:cdr.c__do_reload__1483__&cdr_batch_lock 118: [ 0] 3 0 2277587 759195 UNLOCK:chan_sip.c__restart_monitor__19349__&monlock 293: [ 0] 4 0 3283014 820753 UNLOCK:channel.c__ast_hangup__1675__&chan->lock_dont_use 154: [ 0] 28 0 23266741 830955 UNLOCK:devicestate.c__do_devstate_changes__555__&(&state_changes)->lock 245: [ 0] 2 0 1834371 917185 UNLOCK:rtp.c__ast_rtp_make_compatible__2158__&src->lock_dont_use 206: [ 0] 1 0 925722 925722 UNLOCK:app_minivm.c__load_config__2400__&minivmlock 244: [ 0] 2 0 1883627 941813 UNLOCK:rtp.c__ast_rtp_make_compatible__2157__&dest->lock_dont_use 257: [ 0] 10 0 9635283 963528 UNLOCK:chan_sip.c__handle_request_do__18933__&p->owner->lock_dont_use 94: [ 0] 1 0 995893 995893 UNLOCK:chan_skinny.c__reload_config__6307__&netlock 106: [ 0] 1 0 1213159 1213159 UNLOCK:chan_sip.c__reload_config__21581__&netlock 194: [ 0] 1 0 1785745 1785745 UNLOCK:app_voicemail.c__load_config__10069__&(&users)->lock 180: [ 0] 1 0 1902537 1902537 UNLOCK:chan_agent.c__read_agent_config__1247__&(&agents)->lock 249: [ 0] 2 0 3949848 1974924 UNLOCK:channel.c__ast_call__3522__&chan->lock_dont_use 153: [ 0] 37 0 86017102 2324786 UNLOCK:chan_sip.c__handle_request_do__18936__&netlock 68: [ 0] 1 0 2819754 2819754 UNLOCK:res_odbc.c__odbc_obj_connect__812__&obj->lock 166: [ 0] 1 0 4224003 4224003 UNLOCK:chan_iax2.c__reload_firmware__2127__&(&firmwares)->lock 220: [ 0] 1 0 5489064 5489064 UNLOCK:db.c__ast_db_put__159__&dblock 90: [ 0] 1 0 6034161 6034161 UNLOCK:func_odbc.c__load_module__820__&(&queries)->lock 223: [ 0] 2 0 13618657 6809328 pbx_start_time 174: [ 0] 1 0 9429189 9429189 UNLOCK:chan_dahdi.c__setup_dahdi__14763__&iflock 216: [ 0] 51 0 714982437 14019263 UNLOCK:chan_iax2.c__sched_thread__10101__&sched_lock 33: [ 0] 2 0 143743131 71871565 UNLOCK:loader.c__load_modules__951__&(&module_list)->lock HIGH LOAD: (1166 calls from 10 to 100 cps, mostly at 100cps) 172: [ 0] 1 0 102774 102774 UNLOCK:db.c__ast_db_deltree__134__&dblock 92: [ 0] 3 0 324188 108062 UNLOCK:db.c__ast_db_gettree__528__&dblock 237: [ 0] 6993 0 980913380 140270 UNLOCK:channel.c__ast_write__3224__&chan->lock_dont_use 207: [ 10] 6998 0 132950023 18998 channel_find_locked 114: [ 0] 1 0 215229 215229 UNLOCK:chan_sip.c__sip_send_all_registers__22138__&(®l)->_lock 258: [ 0] 1166 0 270635275 232105 UNLOCK:channel.c__ast_hangup__1675__&chan->lock_dont_use 96: [ 0] 1 0 270764 270764 UNLOCK:chan_skinny.c__restart_monitor__6087__&monlock 124: [ 0] 1 0 281117 281117 UNLOCK:chan_phone.c__restart_monitor__1156__&monlock 57: [ 0] 7001 0 2010859252 287224 UNLOCK:devicestate.c__run_devstate_collector__694__&devstate_collector.lock 135: [ 0] 1 0 352979 352979 UNLOCK:chan_mgcp.c__restart_monitor__3528__&monlock 221: [ 0] 1166 0 436479743 374339 pbx_start_time 178: [ 0] 1 0 379176 379176 UNLOCK:chan_dahdi.c__restart_monitor__8084__&monlock 229: [ 0] 1166 0 447221871 383552 UNLOCK:channel.c____ast_answer__1754__&chan->lock_dont_use 43: [ 0] 1 0 506257 506257 UNLOCK:cdr.c__do_reload__1483__&cdr_batch_lock 70: [ 0] 1 0 535185 535185 UNLOCK:res_odbc.c__odbc_obj_disconnect__768__&obj->lock 241: [ 0] 2332 0 1366088934 585801 UNLOCK:chan_sip.c__handle_request_do__18933__&p->owner->lock_dont_use 130: [ 0] 1 0 686081 686081 UNLOCK:chan_mgcp.c__reload_config__4266__&netlock 7: [ 0] 1 1 690490 690490 utils.c__test_thread_body__242__&test_lock (utils.c:266:test_for_thread_safety:&test_lock[2]) 8: [ 0] 1 0 719418 719418 UNLOCK:utils.c__test_for_thread_safety__274__&test_lock (utils.c:266:test_for_thread_safety:&test_lock[1]) 206: [ 0] 1 0 744944 744944 UNLOCK:app_minivm.c__load_config__2400__&minivmlock 248: [ 0] 1166 0 875115204 750527 UNLOCK:channel.c__ast_queue_frame__1007__&chan->lock_dont_use (chan_sip.c:18893:handle_request_do:&chan->lock_dont_use[1]) 218: [ 0] 35 0 26575870 759310 UNLOCK:chan_dahdi.c__do_monitor__8055__&iflock 9: [ 0] 1 0 1020044 1020044 UNLOCK:utils.c__test_for_thread_safety__279__&test_lock 94: [ 0] 1 0 1029903 1029903 UNLOCK:chan_skinny.c__reload_config__6307__&netlock 10: [ 0] 1 0 1139030 1139030 UNLOCK:utils.c__test_for_thread_safety__283__&test_lock2 106: [ 0] 1 0 1438993 1438993 UNLOCK:chan_sip.c__reload_config__21581__&netlock 118: [ 0] 1 0 1568406 1568406 UNLOCK:chan_sip.c__restart_monitor__19349__&monlock 180: [ 0] 1 0 1682951 1682951 UNLOCK:chan_agent.c__read_agent_config__1247__&(&agents)->lock 194: [ 0] 1 0 1775057 1775057 UNLOCK:app_voicemail.c__load_config__10069__&(&users)->lock 68: [ 0] 1 0 3536200 3536200 UNLOCK:res_odbc.c__odbc_obj_connect__812__&obj->lock 166: [ 0] 1 0 4545482 4545482 UNLOCK:chan_iax2.c__reload_firmware__2127__&(&firmwares)->lock 90: [ 0] 1 0 6601539 6601539 UNLOCK:func_odbc.c__load_module__820__&(&queries)->lock 174: [ 0] 1 0 9218832 9218832 UNLOCK:chan_dahdi.c__setup_dahdi__14763__&iflock 33: [ 0] 2 0 145540500 72770250 UNLOCK:loader.c__load_modules__951__&(&module_list)->lock