Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CRASH] Async with rest client segfaulting when resume route misspelled #3491

Open
volga629-1 opened this issue Oct 8, 2024 · 9 comments
Open

[CRASH] Async with rest client segfaulting when resume route misspelled #3491

volga629-1 opened this issue Oct 8, 2024 · 9 comments
Assignees
@bogdan-iancu
Labels
bug stale
Milestone
3.4.11

Comments

@volga629-1
Copy link

Version

opensips -V
version: opensips 3.4.3 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
unknown revision: unknown
main.c compiled on 00:00:00 Jan 26 2024 with gcc 13

Segfault

root@dev1-msg ~> [/etc/opensips]# coredumpctl list
TIME                            PID UID GID SIG     COREFILE EXE                SIZE
Tue 2023-06-27 09:53:37 EDT 2320686 994 991 SIGSEGV missing  /usr/sbin/opensips    -
Tue 2023-07-11 17:38:43 EDT 2515781 994 991 SIGSEGV missing  /usr/sbin/opensips    -
Mon 2023-08-28 18:18:53 EDT 3004060 994 991 SIGSEGV missing  /usr/sbin/opensips    -
Thu 2023-12-21 07:49:04 EST 3467917 994 991 SIGSEGV missing  /usr/sbin/opensips    -
Sat 2024-07-20 20:57:47 EDT    2802   0   0 SIGSEGV missing  /usr/bin/sngrep       -
Sat 2024-07-20 20:58:42 EDT    2822   0   0 SIGSEGV missing  /usr/bin/sngrep       -
Sun 2024-10-06 17:02:36 EDT  460627 994 991 SIGSEGV present  /usr/sbin/opensips 1.3M
root@dev1-msg ~> [/etc/opensips]# coredumpctl gdb 460627
           PID: 460627 (opensips)
           UID: 994 (opensips)
           GID: 991 (opensips)
        Signal: 11 (SEGV)
     Timestamp: Sun 2024-10-06 17:02:35 EDT (3min 5s ago)
  Command Line: /usr/sbin/opensips -P /run/opensips/opensips.pid -f /etc/opensips/opensips.cfg -m 256 -M 24
    Executable: /usr/sbin/opensips
 Control Group: /system.slice/opensips.service
          Unit: opensips.service
         Slice: system.slice
       Boot ID: 45918117e71b45afadd82112c2e27cc0
    Machine ID: e3e96ddc222643f59a736a3c05e334ed
      Hostname: dev1-msg.networklab.support
       Storage: /var/lib/systemd/coredump/core.opensips.994.45918117e71b45afadd82112c2e27cc0.460627.1728248555000000.zst (present)
  Size on Disk: 1.3M
       Package: opensips/3.4.3-14.fc39
      build-id: a162a39b23649308569ed7b9fe3c17fbb2674fe3
       Message: Process 460627 (opensips) of user 994 dumped core.

                Module libnss_resolve.so.2 from rpm systemd-254.14-1.fc39.x86_64
                Module libcap.so.2 from rpm libcap-2.48-9.fc39.x86_64
                Module libnss_myhostname.so.2 from rpm systemd-254.14-1.fc39.x86_64
                Module textops.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module json.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module libbrotlicommon.so.1 from rpm brotli-1.1.0-1.fc39.x86_64
                Module libbrotlidec.so.1 from rpm brotli-1.1.0-1.fc39.x86_64
                Module libpsl.so.5 from rpm libpsl-0.21.2-4.fc39.x86_64
                Module libssh.so.4 from rpm libssh-0.10.6-2.fc39.x86_64
                Module libnghttp2.so.14 from rpm nghttp2-1.55.1-5.fc39.x86_64
                Module libcurl.so.4 from rpm curl-8.2.1-5.fc39.x86_64
                Module rest_client.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module libhashkit.so.2 from rpm libmemcached-awesome-1.1.4-2.fc39.x86_64
                Module liblua-5.4.so from rpm lua-5.4.6-3.fc39.x86_64
                Module libmemcached.so.11 from rpm libmemcached-awesome-1.1.4-2.fc39.x86_64
                Module libmariadb.so.3 from rpm mariadb-connector-c-3.3.8-1.fc39.x86_64
                Module lua.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module proto_smpp.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module libhiredis.so.1.0.0 from rpm hiredis-1.0.2-5.fc39.x86_64
                Module cachedb_redis.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module cachedb_local.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module msilo.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module avpops.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module xcap.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module pua_dialoginfo.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module pua.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module presence_callinfo.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module presence_dialoginfo.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module presence_xml.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module presence_mwi.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module liblzma.so.5 from rpm xz-5.4.4-1.fc39.x86_64
                Module libxml2.so.2 from rpm libxml2-2.10.4-3.fc39.x86_64
                Module presence.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module dialog.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module freeswitch.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module libjson-c.so.5 from rpm json-c-0.17-1.fc39.x86_64
                Module cgrates.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module acc.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module nat_traversal.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module nathelper.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module domain.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module auth_db.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module auth.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module registrar.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module usrloc.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module uac.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module uac_auth.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module alias_db.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module group.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module mi_fifo.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module event_stream.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module mi_http.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module libffi.so.8 from rpm libffi-3.4.4-4.fc39.x86_64
                Module libgmp.so.10 from rpm gmp-6.2.1-5.fc39.x86_64
                Module libhogweed.so.6 from rpm nettle-3.9.1-2.fc39.x86_64
                Module libnettle.so.8 from rpm nettle-3.9.1-2.fc39.x86_64
                Module libtasn1.so.6 from rpm libtasn1-4.19.0-3.fc39.x86_64
                Module libunistring.so.5 from rpm libunistring-1.1-5.fc39.x86_64
                Module libidn2.so.0 from rpm libidn2-2.3.7-1.fc39.x86_64
                Module libp11-kit.so.0 from rpm p11-kit-0.25.5-1.fc39.x86_64
                Module libgnutls.so.30 from rpm gnutls-3.8.6-1.fc39.x86_64
                Module libmicrohttpd.so.12 from rpm libmicrohttpd-1.0.1-1.fc39.x86_64
                Module httpd.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module sipmsgops.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module maxfwd.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module path.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module rr.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module tm.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module sl.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module signaling.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module libpcre2-8.so.0 from rpm pcre2-10.42-1.fc39.2.x86_64
                Module libcrypt.so.2 from rpm libxcrypt-4.4.36-2.fc39.x86_64
                Module libselinux.so.1 from rpm libselinux-3.5-5.fc39.x86_64
                Module libsasl2.so.3 from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
                Module libevent-2.1.so.7 from rpm libevent-2.1.12-9.fc39.x86_64
                Module libkeyutils.so.1 from rpm keyutils-1.6.3-1.fc39.x86_64
                Module libkrb5support.so.0 from rpm krb5-1.21.3-1.fc39.x86_64
                Module libcom_err.so.2 from rpm e2fsprogs-1.47.0-2.fc39.x86_64
                Module libk5crypto.so.3 from rpm krb5-1.21.3-1.fc39.x86_64
                Module libkrb5.so.3 from rpm krb5-1.21.3-1.fc39.x86_64
                Module libz.so.1 from rpm zlib-1.2.13-4.fc39.x86_64
                Module libgssapi_krb5.so.2 from rpm krb5-1.21.3-1.fc39.x86_64
                Module libcrypto.so.3 from rpm openssl-3.1.1-4.fc39.x86_64
                Module libssl.so.3 from rpm openssl-3.1.1-4.fc39.x86_64
                Module libpq.so.5 from rpm libpq-15.3-1.fc39.x86_64
                Module db_postgres.so from rpm opensips-3.4.3-14.fc39.x86_64
                Module opensips from rpm opensips-3.4.3-14.fc39.x86_64
                Stack trace of thread 460627:
                #0  0x000055c13b4c35cf run_top_route (opensips + 0x2e5cf)
                #1  0x00007f1f1f0e9f5f run_resume_route (tm.so + 0x14f5f)
                #2  0x000055c13b4c0ef8 do_action (opensips + 0x2bef8)
                #3  0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #4  0x000055c13b4c2ae9 run_actions (opensips + 0x2dae9)
                #5  0x000055c13b4c1751 do_action (opensips + 0x2c751)
                #6  0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #7  0x000055c13b4c1560 do_action (opensips + 0x2c560)
                #8  0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #9  0x000055c13b4c1560 do_action (opensips + 0x2c560)
                #10 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #11 0x000055c13b4c2ae9 run_actions (opensips + 0x2dae9)
                #12 0x000055c13b4c1751 do_action (opensips + 0x2c751)
                #13 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #14 0x000055c13b4c1560 do_action (opensips + 0x2c560)
                #15 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #16 0x000055c13b4c1560 do_action (opensips + 0x2c560)
                #17 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #18 0x000055c13b4c2ae9 run_actions (opensips + 0x2dae9)
                #19 0x000055c13b4c1751 do_action (opensips + 0x2c751)
                #20 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #21 0x000055c13b4c1560 do_action (opensips + 0x2c560)
                #22 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #23 0x000055c13b4c1560 do_action (opensips + 0x2c560)
                #24 0x000055c13b4c2a50 run_action_list (opensips + 0x2da50)
                #25 0x000055c13b4c34ea run_actions (opensips + 0x2e4ea)
                #26 0x000055c13b5328e6 receive_msg (opensips + 0x9d8e6)
                #27 0x000055c13b6658b3 tcp_handle_req (opensips + 0x1d08b3)
                #28 0x000055c13b64fe70 handle_io (opensips + 0x1bae70)
                #29 0x000055c13b650f58 io_wait_loop_epoll (opensips + 0x1bbf58)
                #30 0x000055c13b652f29 tcp_start_processes (opensips + 0x1bdf29)
                #31 0x000055c13b4b8534 main_loop (opensips + 0x23534)
                #32 0x00007f1f312f914a __libc_start_call_main (libc.so.6 + 0x2814a)
                #33 0x00007f1f312f920b __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2820b)
                #34 0x000055c13b4b8c25 _start (opensips + 0x23c25)
                ELF object binary architecture: AMD x86-64

GNU gdb (Fedora Linux) 14.2-1.fc39
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/opensips...
Reading symbols from /usr/lib/debug/usr/sbin/opensips-3.4.3-14.fc39.x86_64.debug...

warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
[New LWP 460627]

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n])
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
Missing separate debuginfo for /usr/lib64/opensips/modules/db_postgres.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/httpd.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/presence.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/presence_mwi.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/presence_xml.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/presence_dialoginfo.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/presence_callinfo.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/pua.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/pua_dialoginfo.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/xcap.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/cachedb_redis.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/proto_smpp.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/lua.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/rest_client.so.
The debuginfo package for this file is probably broken.
Missing separate debuginfo for /usr/lib64/opensips/modules/json.so.
The debuginfo package for this file is probably broken.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/sbin/opensips -P /run/opensips/opensips.pid -f /etc/opensips/opensips.cfg'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055c13b4c35cf in run_top_route (sr=..., msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:245
245		        (REQUEST_ROUTE|ONREPLY_ROUTE) && !strcmp(sr.name, "0")))
Missing separate debuginfos, use: dnf debuginfo-install cyrus-sasl-lib-2.1.28-11.fc39.x86_64 glibc-2.38-18.fc39.x86_64 gmp-6.--Type <RET> for more, q to quit, c to continue without paging--c
2.1-5.fc39.x86_64 gnutls-3.8.6-1.fc39.x86_64 json-c-0.17-1.fc39.x86_64 keyutils-libs-1.6.3-1.fc39.x86_64 krb5-libs-1.21.3-1.fc39.x86_64 libcap-2.48-9.fc39.x86_64 libcom_err-1.47.0-2.fc39.x86_64 libcurl-8.2.1-5.fc39.x86_64 libevent-2.1.12-9.fc39.x86_64 libffi-3.4.4-4.fc39.x86_64 libgcc-13.3.1-1.fc39.x86_64 libidn2-2.3.7-1.fc39.x86_64 libmemcached-awesome-1.1.4-2.fc39.x86_64 libmicrohttpd-1.0.1-1.fc39.x86_64 libnghttp2-1.55.1-5.fc39.x86_64 libpq-15.3-1.fc39.x86_64 libpsl-0.21.2-4.fc39.x86_64 libselinux-3.5-5.fc39.x86_64 libssh-0.10.6-2.fc39.x86_64 libstdc++-13.3.1-1.fc39.x86_64 libtasn1-4.19.0-3.fc39.x86_64 libunistring-1.1-5.fc39.x86_64 libxcrypt-4.4.36-2.fc39.x86_64 libxml2-2.10.4-3.fc39.x86_64 lua-libs-5.4.6-3.fc39.x86_64 mariadb-connector-c-3.3.8-1.fc39.x86_64 nettle-3.9.1-2.fc39.x86_64 openldap-2.6.7-1.fc39.x86_64 openssl-libs-3.1.1-4.fc39.x86_64 p11-kit-0.25.5-1.fc39.x86_64 pcre2-10.42-1.fc39.2.x86_64 zlib-1.2.13-4.fc39.x86_64
(gdb) bt
#0  0x000055c13b4c35cf in run_top_route (sr=..., msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:245
#1  0x00007f1f1f0e9f5f in run_resume_route (resume_route=0x7f1f2fa572e0, run_post_cb=1, msg=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/modules/tm/async.c:61
#2  t_handle_async (msg=<optimized out>, a=<optimized out>, resume_route=0x7f1f2fa572e0, timeout=<optimized out>,
    params=<optimized out>) at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/modules/tm/async.c:407
#3  0x000055c13b4c0ef8 in do_action (a=a@entry=0x7f1f2fa57348, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:1081
#4  0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#5  0x000055c13b4c2ae9 in run_actions (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
#6  0x000055c13b4c1751 in do_action (a=a@entry=0x7f1f2fa45ec8, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:803
#7  0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#8  0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa45ff8, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
#9  0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#10 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa46128, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
#11 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#12 0x000055c13b4c2ae9 in run_actions (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
#13 0x000055c13b4c1751 in do_action (a=a@entry=0x7f1f2fa3a738, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:803
#14 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#15 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa3a868, msg=msg@entry=0x7f1f2fdf4a30)
--Type <RET> for more, q to quit, c to continue without paging--  c
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
#16 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#17 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa3b248, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
#18 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#19 0x000055c13b4c2ae9 in run_actions (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
#20 0x000055c13b4c1751 in do_action (a=a@entry=0x7f1f2fa28810, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:803
#21 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#22 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa28940, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
#23 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#24 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa28a70, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
#25 0x000055c13b4c2a50 in run_action_list (a=a@entry=0x7f1f2fa1b7a0, msg=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
#26 0x000055c13b4c34ea in run_actions (msg=<optimized out>, a=0x7f1f2fa1b7a0)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
#27 run_top_route (sr=..., msg=<optimized out>) at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:250
#28 0x000055c13b5328e6 in receive_msg (buf=<optimized out>, len=<optimized out>, rcv_info=<optimized out>,
    existing_context=<optimized out>, msg_flags=<optimized out>)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/receive.c:225
#29 0x000055c13b6658b3 in tcp_handle_req (_parallel_handling=<optimized out>, _max_msg_chunks=<optimized out>,
    con=0x7f1f20042fc0, req=0x55c13b87b3c0 <tcp_current_req>) at net/proto_tcp/tcp_common.h:423
#30 tcp_read_req (con=0x7f1f20042fc0, bytes_read=0x7ffe879a6f74) at net/proto_tcp/proto_tcp.c:738
#31 0x000055c13b64fe70 in handle_io (fm=<optimized out>, idx=idx@entry=2, event_type=event_type@entry=1)
    at net/net_tcp_proc.c:329
#32 0x000055c13b650f58 in io_wait_loop_epoll (h=<optimized out>, t=<optimized out>, repeat=<optimized out>)
    at net/../io_wait_loop.h:305
#33 tcp_worker_proc_loop () at net/net_tcp_proc.c:442
#34 0x000055c13b652f29 in tcp_start_processes (chd_rank=0x55c13b74bdc4 <chd_rank>, startup_done=0x7f1f20041918)
    at net/net_tcp.c:2119
#35 0x000055c13b4b8534 in main_loop () at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/main.c:243
#36 main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/main.c:966
(gdb) bt full
#0  0x000055c13b4c35cf in run_top_route (sr=..., msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:245
        bk_action_flags = 0
        route_stack_start_bkp = <optimized out>
        route_stack_size_bkp = <optimized out>
        ret = <optimized out>
        ctx = 0x0
        __FUNCTION__ = "run_top_route"
        recursing = 1
#1  0x00007f1f1f0e9f5f in run_resume_route (resume_route=0x7f1f2fa572e0, run_post_cb=1, msg=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/modules/tm/async.c:61
No locals.
#2  t_handle_async (msg=<optimized out>, a=<optimized out>, resume_route=0x7f1f2fa572e0, timeout=<optimized out>,
    params=<optimized out>) at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/modules/tm/async.c:407
        ctx = <optimized out>
        t = <optimized out>
        r = <optimized out>
        fd = <optimized out>
        failure = <optimized out>
        sync = <optimized out>
        resume = <optimized out>
        __FUNCTION__ = "t_handle_async"
#3  0x000055c13b4c0ef8 in do_action (a=a@entry=0x7f1f2fa57348, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:1081
        ret = 0
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging-- c
        spec = <optimized out>
        val = {rs = {s = 0x0, len = 0}, ri = 996920488, flags = 21953}
        start = {tv_sec = 18, tv_usec = 139771920012744}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = 0x7f1f1e1afec0
        cmdp = {0x7ffe879a5478, 0x7f1f2fa56df0, 0x0, 0x7f1f2fa56fb0, 0x0, 0x0, 0x0,
          0x7f1f313cd635 <regexec@@GLIBC_2.3.4+69>}
        tmp_vals = {{rs = {s = 0x7ffe879a54b0 "\020U\232\207\376\177", len = 995738127}, ri = 0, flags = 0}, {rs = {
              s = 0x7f1f2fa9f4c0 "http://apilayer.net/api/validate?access_key=40c952e7f57363b5aa10bd2298c11c67&number=<null>", len = 90}, ri = 803171832, flags = 32543}, {rs = {s = 0x7ffe879a5570 "\370m\337/\037\177", len = 799774392}, ri = 36,
            flags = 0}, {rs = {s = 0x15 <error: Cannot access memory at address 0x15>, len = -2019928816}, ri = 995319691,
            flags = 21953}, {rs = {s = 0x7ffe879a5560 "\240U\232\207\376\177", len = -2019928800}, ri = 29, flags = 0}, {
            rs = {s = 0xf6e2275a60c3da00 <error: Cannot access memory at address 0xf6e2275a60c3da00>, len = -2019928736},
            ri = 0, flags = 0}, {rs = {s = 0x7 <error: Cannot access memory at address 0x7>, len = 0}, ri = 803162672,
            flags = 32543}, {rs = {s = 0x7f1f2fa44f88 "\002", len = -2019928544}, ri = 995341248, flags = 21953}}
        route_p = <optimized out>
        sval = {s = 0x7f1f2fdf4a30 "\002", len = 0}
        __FUNCTION__ = "do_action"
#4  0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa57348
#5  0x000055c13b4c2ae9 in run_actions (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        __FUNCTION__ = "run_actions"
#6  0x000055c13b4c1751 in do_action (a=a@entry=0x7f1f2fa45ec8, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f1f2fdf4a30 "\002", len = 799292224}, ri = 0, flags = 0}
        start = {tv_sec = 140731173459728, tv_usec = 139771946784377}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x3, 0x15, 0x7ffe879a5770, 0x55c13b535f8b <comp_s2s+1259>, 0x7ffe00000001, 0x0, 0x7f1f1f0b9b70 <cmds+1680>,
          0xf6e2275a60c3da00}
        tmp_vals = {{rs = {s = 0x7ffe879a5850 "\370\214\253/\037\177", len = 0}, ri = 7, flags = 0}, {rs = {s = 0x0,
              len = 803162672}, ri = 799292224, flags = 32543}, {rs = {s = 0x7ffe879a5880 "\240Z\232\207\376\177",
              len = 995341248}, ri = 892549682, flags = 1397709600}, {rs = {
              s = 0x532d3035322d5049 <error: Cannot access memory at address 0x532d3035322d5049>, len = 1565544525},
            ri = 926037552, flags = 1310734941}, {rs = {
              s = 0x535b3a454349544f <error: Cannot access memory at address 0x535b3a454349544f>, len = 1381978957},
            ri = 0, flags = 32543}, {rs = {
              s = 0x3130313000000015 <error: Cannot access memory at address 0x3130313000000015>, len = 544175136},
            ri = 803162672, flags = 32543}, {rs = {
              s = 0x55c13b87b461 <tcp_current_req+161> "[email protected]>;tag=6A446C3C629934BC4F569C9D684D5874\r\nAllow: ACK, BYE, CANCEL, INFO, INVITE, MESSAGE, NOTIFY, OPTIONS, REFER\r\nSupported: replaces, path\r\nTo: <sip:[email protected]>\r\nCall-"..., len = 5}, ri = 0, flags = 0}, {rs = {
              s = 0x3430353300000000 <error: Cannot access memory at address 0x3430353300000000>, len = 998749281}, ri = 5,
            flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x55c13b68f97c "unknown", len = 7}
        __FUNCTION__ = "do_action"
#7  0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa45ec8
#8  0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa45ff8, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
        ret = 1
        v = 1
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7ffe879a5980 "0J\337/\037\177", len = 1}, ri = 996920488, flags = 21953}
        start = {tv_sec = 0, tv_usec = 139771920481336}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x55c13b6bcca8, 0x7f1f2fa3f718, 0x7f1f2fdf4a30, 0x55c13b749f00 <_oser_err_info>, 0x7f1f2fdf4a30,
          0x7f1f2fa3f848, 0x7ffe879a59d0, 0x55c13b4c2a50 <run_action_list+80>}
        tmp_vals = {{rs = {s = 0x3000000030 <error: Cannot access memory at address 0x3000000030>, len = -2019927320},
            ri = -2019927584, flags = 32766}, {rs = {s = 0x7f1f2fa46128 "\006", len = 2}, ri = 0, flags = 0}, {rs = {
              s = 0x7ffe879a5ae0 "", len = 995337748}, ri = -2019927488, flags = 32766}, {rs = {
              s = 0x7f1f313a064e <__tzfile_compute+494> "\205\300\017\205\004\004", len = 205}, ri = 0, flags = 0}, {rs = {
              s = 0x7ffe879a5aa0 "\340Z\232\207\376\177", len = -2019927456}, ri = 29, flags = 0}, {rs = {
              s = 0x7ffe879a5aa0 "\340Z\232\207\376\177", len = -2019927392}, ri = 1, flags = 0}, {rs = {
              s = 0x6702faeb <error: Cannot access memory at address 0x6702faeb>, len = -2019927264}, ri = -2019927408,
            flags = 32766}, {rs = {s = 0x7f1f3139f3d0 <__tz_convert+176> "\205\300\017\204\200", len = -2019927408},
            ri = 825873416, flags = 32543}}
        route_p = <optimized out>
        sval = {s = 0x1 <error: Cannot access memory at address 0x1>, len = 1623448064}
        __FUNCTION__ = "do_action"
#9  0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa45ff8
#10 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa46128, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
        ret = 1
        v = 0
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7ffe879a5be0 "\240\314k;\301U", len = 1}, ri = 996920488, flags = 21953}
        start = {tv_sec = 139771920482824, tv_usec = 140731173461232}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x55c13b6bcca8, 0x291e, 0x500000000, 0x7f1f3132dc66 <snprintf+150>, 0x55c13b6bcca0, 0x0, 0x7f1f200434e0,
          0x7}
        tmp_vals = {{rs = {s = 0x3000000030 <error: Cannot access memory at address 0x3000000030>, len = 1623448064},
            ri = -2019926976, flags = 32766}, {rs = {s = 0x7f1f2fab81f0 "\004", len = -2019926800}, ri = 803169856,
            flags = 32543}, {rs = {s = 0x7f1f2fdf4a30 "\002", len = 799627413}, ri = -2019926880, flags = 32766}, {rs = {
              s = 0x55c13b524d8e <pv_get_spec_value+174> "\205\300\017\205", <incomplete sequence \362>, len = 205},
            ri = 995711567, flags = 21953}, {rs = {s = 0x7ffe879a5d00 "@]\232\207\376\177", len = 1623448064}, ri = 29,
            flags = 0}, {rs = {s = 0x7f1f2fa3a590 "\210\201\253/\037\177", len = 21}, ri = 1623448064, flags = -152950950},
          {rs = {s = 0x7f1f2fab81e0 "\230\244\243/\037\177", len = -2019926280}, ri = 0, flags = 0}, {rs = {
              s = 0x7ffe879a5f10 "\200X\251/\037\177", len = 803162672}, ri = 996975800, flags = 21953}}
        route_p = <optimized out>
        sval = {s = 0x7ffe879a5b80 "\340[\232\207\376\177", len = 1623448064}
        __FUNCTION__ = "do_action"
#11 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa46128
#12 0x000055c13b4c2ae9 in run_actions (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        __FUNCTION__ = "run_actions"
#13 0x000055c13b4c1751 in do_action (a=a@entry=0x7f1f2fa3a738, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f1f2fdf4a30 "\002", len = 799245224}, ri = 0, flags = 0}
        start = {tv_sec = 140731173461680, tv_usec = 139771946784377}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x7ffe879a5ef8, 0x7ffe879a5f10, 0x0, 0x55c13b535f8b <comp_s2s+1259>, 0x7ffe879a5f00,
          0x55c13b58de14 <destroy_index_avp+84>, 0x55c157fb0a80, 0xf6e2275a60c3da00}
        tmp_vals = {{rs = {s = 0x2 <error: Cannot access memory at address 0x2>, len = 0}, ri = 7, flags = 0}, {rs = {
              s = 0x7f1f2fa90a60 "043f04400438043204350442", len = 24}, ri = 799245224, flags = 32543}, {rs = {
              s = 0x7f1f2fa95880 "text/plain; charset=[UTF16BE]", len = 29}, ri = 14898, flags = 0}, {rs = {
              s = 0x200000000 <error: Cannot access memory at address 0x200000000>, len = 1565544525}, ri = 1623448064,
            flags = -152950950}, {rs = {s = 0x525b3a454349544f <error: Cannot access memory at address 0x525b3a454349544f>,
              len = -2019926096}, ri = 0, flags = 32766}, {rs = {s = 0x15 <error: Cannot access memory at address 0x15>,
              len = 803162672}, ri = 799240960, flags = 32543}, {rs = {
              s = 0x55c13b87b461 <tcp_current_req+161> "[email protected]>;tag=6A446C3C629934BC4F569C9D684D5874\r\nAllow: ACK, BYE, CANCEL, INFO, INVITE, MESSAGE, NOTIFY, OPTIONS, REFER\r\nSupported: replaces, path\r\nTo: <sip:[email protected]>\r\nCall-"..., len = 5}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 998749281}, ri = 5, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x55c13b68f97c "unknown", len = 7}
        __FUNCTION__ = "do_action"
#14 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa3a738
#15 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa3a868, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
        ret = 1
        v = 1
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x3732363036340002 <error: Cannot access memory at address 0x3732363036340002>, len = 1623448064},
          ri = 5, flags = 0}
        start = {tv_sec = 140731173463472, tv_usec = 1}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x7ffe879a61d0, 0x7f1f31353e81 <__vsnprintf_internal+145>, 0x7ffe879a63d0, 0x7ffe879a63ff, 0x7ffe879a67d0,
          0x0, 0x2, 0xf6e2275a60c3da00}
        tmp_vals = {{rs = {s = 0x353300000000 <error: Cannot access memory at address 0x353300000000>, len = 799257160},
            ri = 3, flags = 0}, {rs = {s = 0x0, len = 803162672}, ri = 799239512, flags = 32543}, {rs = {
              s = 0x7ffe879a6280 "\240d\232\207\376\177", len = 995337036}, ri = 0, flags = 0}, {rs = {
              s = 0x500000000 <error: Cannot access memory at address 0x500000000>, len = 5}, ri = 5, flags = 5}, {rs = {
              s = 0x500000005 <error: Cannot access memory at address 0x500000005>, len = 5}, ri = 0, flags = 0}, {rs = {
              s = 0x55c15802f928 "", len = 16777217}, ri = 1, flags = 0}, {rs = {s = 0x55c15802f880 "\320M\002X\301U",
              len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x7ffe879a65c0 "|\371h;\301U", len = -11}
        __FUNCTION__ = "do_action"
#16 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa3a868
#17 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa3b248, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
        ret = 1
        v = 1
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f1f2fa27e68 "\005", len = 803162672}, ri = 799178368, flags = 32543}
        start = {tv_sec = 140731173462848, tv_usec = 94288412254686}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x7ffe879a63b0, 0xf6e2275a60c3da00, 0x7ffe879a63a0, 0x7ffe879a64d0, 0x6, 0x7ffe00000015, 0x7f1f2fdf4a30,
          0x7f1f2fa25ed0}
        tmp_vals = {{rs = {s = 0x7f1f2fa27db5 "P-hint)", len = 6}, ri = 0, flags = 4}, {rs = {
              s = 0xf6e2275a60c3da00 <error: Cannot access memory at address 0xf6e2275a60c3da00>, len = 998749281},
            ri = 799178344, flags = 32543}, {rs = {s = 0x7ffe879a64d0 "\n", len = 0}, ri = 803162672, flags = 32543}, {
            rs = {s = 0x7f1f2fa27fc0 "\002", len = -2019924912}, ri = 995249550, flags = 21953}, {rs = {
              s = 0x7ffe879a64a0 "\340d\232\207\376\177", len = 1623448064}, ri = 29, flags = 0}, {rs = {
              s = 0xf6e2275a60c3da00 <error: Cannot access memory at address 0xf6e2275a60c3da00>, len = 18},
            ri = 799180328, flags = 32543}, {rs = {s = 0x7 <error: Cannot access memory at address 0x7>, len = 0},
            ri = 803162672, flags = 32543}, {rs = {s = 0x7f1f2fa27fc0 "\002", len = -2019924640}, ri = 995337205,
            flags = 21953}}
        route_p = <optimized out>
        sval = {s = 0x7f1f2fdf4a30 "\002", len = 799178688}
        __FUNCTION__ = "do_action"
#18 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa3b248
#19 0x000055c13b4c2ae9 in run_actions (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        __FUNCTION__ = "run_actions"
#20 0x000055c13b4c1751 in do_action (a=a@entry=0x7f1f2fa28810, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {
            s = 0x55c13b87b508 <tcp_current_req+328> "[email protected]>\r\nCall-ID: CF6F92A4BB273663AE99157FAEC8B964D5FECAD6\r\nContent-Type: text/plain;charset=utf-8\r\nCSeq: 1 MESSAGE\r\nProxy-Authorization: Digest username=\"10101\",realm=\"networklab.c"..., len = 11}, ri = 0, flags = 0}
        start = {tv_sec = 139771923876400, tv_usec = -656919326518683136}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x7f1f2fa8d508, 0xf, 0x0, 0x7f1f2fdf4a30, 0x7ffe879a6bc0, 0xf6e2275a60c3da00, 0x0, 0xf6e2275a60c3da00}
        tmp_vals = {{rs = {s = 0x7f1f2fa22350 "\t", len = 799181120}, ri = 1, flags = 0}, {rs = {s = 0x0, len = 803162672},
            ri = 799156968, flags = 32543}, {rs = {s = 0x7ffe879a67c0 "\340i\232\207\376\177", len = 995337004}, ri = 1,
            flags = 32766}, {rs = {s = 0xf6e2275a60c3da00 <error: Cannot access memory at address 0xf6e2275a60c3da00>,
              len = -2019924096}, ri = 1623448064, flags = -152950950}, {rs = {
              s = 0x2 <error: Cannot access memory at address 0x2>, len = -2019924144}, ri = 520854880, flags = 32543}, {
            rs = {s = 0x12 <error: Cannot access memory at address 0x12>, len = 803162672}, ri = 803162672, flags = 32543},
          {rs = {s = 0x7ffe879a67c0 "\340i\232\207\376\177", len = 994825682}, ri = 803162672, flags = 875704370}, {rs = {
              s = 0xf6e2275a60c3da00 <error: Cannot access memory at address 0xf6e2275a60c3da00>, len = 6},
            ri = -2019924112, flags = 32766}}
        route_p = <optimized out>
        sval = {s = 0x55c13b68f97c "unknown", len = 7}
        __FUNCTION__ = "do_action"
#21 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa28810
#22 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa28940, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
        ret = 1
        v = 0
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x55c13b749dc0 <int2str_buf+64> "1", len = 1}, ri = 799149312, flags = 32543}
        start = {tv_sec = 140731173464160, tv_usec = 139771946130534}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x7ffe879a6900, 0x7f1f2fa22348, 0x7f1f2fdf4a30, 0x55c13b749f00 <_oser_err_info>, 0x7f1f2fdf4a30,
          0x7f1f2fa22478, 0x7ffe879a6910, 0x55c13b4c2a50 <run_action_list+80>}
        tmp_vals = {{rs = {s = 0x7ffe879a6960 "\344%m;\301U", len = 1}, ri = 1, flags = 0}, {rs = {
              s = 0x7f1f2fa28a70 "\006", len = 2}, ri = 0, flags = 0}, {rs = {s = 0x7ffe879a6a20 "@l\232\207\376\177",
              len = 995337748}, ri = 17, flags = 0}, {rs = {s = 0x2 <error: Cannot access memory at address 0x2>,
              len = 35}, ri = 2024, flags = 0}, {rs = {s = 0x55c13b74a300 <ctime_buf> "Sun Oct  6 17:02:35",
              len = 997008868}, ri = -2019923536, flags = 32766}, {rs = {
              s = 0x1 <error: Cannot access memory at address 0x1>, len = 997008868}, ri = 65, flags = 0}, {rs = {
              s = 0x7f1f2fa8f9b0 "[DBALIASES_USER_MAP] Formated User sip:[email protected]\n",
              len = -2019923488}, ri = -2019923424, flags = 32766}, {rs = {
              s = 0x55c13b590727 <xlog_1+343> "H\213\025\"i\033", len = 996813086}, ri = 997499652, flags = 21953}}
        route_p = <optimized out>
        sval = {s = 0x55c13b6bcca8 "%sNOTICE:%.*s", len = 996813086}
        __FUNCTION__ = "do_action"
#23 0x000055c13b4c2a50 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa28940
#24 0x000055c13b4c1560 in do_action (a=a@entry=0x7f1f2fa28a70, msg=msg@entry=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:836
        ret = 1
        v = 1
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7ffe879a6b60 "", len = 996147021}, ri = 13088, flags = 0}
        start = {tv_sec = 139771657784464, tv_usec = 94288415798322}
        end_time = <optimized out>
        cmd = <optimized out>
        acmd = <optimized out>
        cmdp = {0x55c13b87b586 <tcp_current_req+454>, 0x55c13b87b58e <tcp_current_req+462>,
          0x55c13b87b719 <tcp_current_req+857>, 0x7f1f2fdf5cf8, 0x7ffe879a6b60, 0xf6e2275a60c3da00,
          0x55c13b74a31a <ctime_buf+26>, 0x55c13b87b6ee <tcp_current_req+814>}
        tmp_vals = {{rs = {s = 0x7f1f2fdf5f68 "\r", len = 13}, ri = 803168104, flags = 32543}, {rs = {
              s = 0x55c13b87b719 <tcp_current_req+857> "", len = -2019922944}, ri = 995934341, flags = 21953}, {rs = {
              s = 0x1 <error: Cannot access memory at address 0x1>, len = 1}, ri = 180, flags = 0}, {rs = {
              s = 0x31 <error: Cannot access memory at address 0x31>, len = -2019922816}, ri = 995711567, flags = 21953}, {
            rs = {s = 0x1 <error: Cannot access memory at address 0x1>, len = 803167480}, ri = 3, flags = 0}, {rs = {
              s = 0xc879a6c10 <error: Cannot access memory at address 0xc879a6c10>, len = -2019922640}, ri = 1623448064,
            flags = -152950950}, {rs = {s = 0x7f1f1e2ded48 "%sINFO:proto_smpp:%s: Successfully bound transceiver \"%s\"\n",
              len = -1}, ri = 998749963, flags = 21953}, {rs = {
              s = 0x55c13b6c8fe8 "\340D\360\377\210C\360\3778E\360\377\360M\360\377xF\360\377`F\360\377\020G\360\377\270F\360\377hC\360\377\230F\360\377(G\360\377\330F\360\377xG\360\377\260G\360\377\020K\360\377PI\360\377PL\360\377\220H\360\377\260K\360\377PJ\360\377\020M\360\377\020H\360\377PK\360\377\320I\360\377\220L\360\377\360H\360\377\020L\360\377\260J\360\377\220M\360\377\320G\360\3770K\360\377\220I\360\377pL\360\377\320H\360\377\360K\360\377\220J\360\377PM\360\377PH\360\377\220K\360\377\210C\360\377\020J\360\377\320L\360\377\020I\360\3770L\360\377\320J\360\377\260M\360\377\210C\360\377\210C\360\377\210C\360\377\210C\360\377"..., len = 803168360}, ri = 803162672, flags = 32543}}
        route_p = <optimized out>
        sval = {s = 0x55c13b68f97c "unknown", len = 7}
        __FUNCTION__ = "do_action"
#25 0x000055c13b4c2a50 in run_action_list (a=a@entry=0x7f1f2fa1b7a0, msg=0x7f1f2fdf4a30)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:190
        ret = -1
        t = 0x7f1f2fa28a70
#26 0x000055c13b4c34ea in run_actions (msg=<optimized out>, a=0x7f1f2fa1b7a0)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = <optimized out>
        error = <optimized out>
#27 run_top_route (sr=..., msg=<optimized out>) at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/action.c:250
        bk_action_flags = 0
        route_stack_start_bkp = <optimized out>
        route_stack_size_bkp = <optimized out>
        ret = <optimized out>
        ctx = 0x0
        __FUNCTION__ = "run_top_route"
        recursing = 1
#28 0x000055c13b5328e6 in receive_msg (buf=<optimized out>, len=<optimized out>, rcv_info=<optimized out>,
    existing_context=<optimized out>, msg_flags=<optimized out>)
    at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/receive.c:225
        msg = 0x7f1f2fdf4a30
        start = {tv_sec = 0, tv_usec = 94288413586684}
        rc = <optimized out>
        old_route_type = <optimized out>
        tmp = <optimized out>
        in_buff = {
          s = 0x55c13b87b3c0 <tcp_current_req> "MESSAGE sip:[email protected] SIP/2.0\r\nVia: SIP/2.0/TCP mysocketip:myport;branch=z9hG4bK36c2.ac69fa06.0\r\nMax-Forwards: 68\r\nFrom: \"Slava Bendersky\" <sip:[email protected]>;tag=6A446C3C629934"..., len = 857}
        ctx = 0x7f1f2fdf5d60
        __FUNCTION__ = "receive_msg"
#29 0x000055c13b6658b3 in tcp_handle_req (_parallel_handling=<optimized out>, _max_msg_chunks=<optimized out>,
    con=0x7f1f20042fc0, req=0x55c13b87b3c0 <tcp_current_req>) at net/proto_tcp/tcp_common.h:423
        local_rcv = {src_ip = {af = 2, len = 4, u = {addrl = {3383437110, 0}, addr32 = {3383437110, 0, 0, 0}, addr16 = {
                10038, 51627, 0, 0, 0, 0, 0, 0}, addr = "6'\253\311", '\000' <repeats 11 times>}}, dst_ip = {af = 2,
            len = 4, u = {addrl = {4201727390, 0}, addr32 = {4201727390, 0, 0, 0}, addr16 = {17822, 64113, 0, 0, 0, 0, 0,
                0}, addr = "\236Eq\372", '\000' <repeats 11 times>}}, src_port = 2263, dst_port = 5084, proto = 2,
          proto_reserved1 = 1785003235, proto_reserved2 = 0, src_su = {s = {sa_family = 2,
              sa_data = "\b\3276'\253\311\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 55048,
              sin_addr = {s_addr = 3383437110}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2,
              sin6_port = 55048, sin6_flowinfo = 3383437110, sin6_addr = {__in6_u = {
                  __u6_addr8 = "\000\000\000\000\000\000\000\000\340o\232\207\376\177\000", __u6_addr16 = {0, 0, 0, 0,
                    28640, 34714, 32766, 0}, __u6_addr32 = {0, 0, 2275045344, 32766}}}, sin6_scope_id = 16}},
          bind_address = 0x7f1f2fa0b558}
        msg_buf = 0x55c13b87b3c0 <tcp_current_req> "MESSAGE sip:[email protected] SIP/2.0\r\nVia: SIP/2.0/TCP mysocketip:myport;branch=z9hG4bK36c2.ac69fa06.0\r\nMax-Forwards: 68\r\nFrom: \"Slava Bendersky\" <sip:[email protected]>;tag=6A446C3C629934"...
        msg_len = 857
        c = 0 '\000'
        size = 0
        msg_buf_cpy = 0x0
        error = <optimized out>
        local_rcv = <optimized out>
        msg_buf = <optimized out>
        msg_len = <optimized out>
        size = <optimized out>
        c = <optimized out>
        msg_buf_cpy = <optimized out>
        __FUNCTION__ = <optimized out>
        error = <optimized out>
        _timeout = <optimized out>
#30 tcp_read_req (con=0x7f1f20042fc0, bytes_read=0x7ffe879a6f74) at net/proto_tcp/proto_tcp.c:738
        bytes = <optimized out>
        total_bytes = <optimized out>
        req = <optimized out>
        src_su = {s = {sa_family = 64234,
            sa_data = "\002g\000\000\000\000\000\332\303`Z'\342", <incomplete sequence \366>}, sin = {sin_family = 64234,
            sin_port = 26370, sin_addr = {s_addr = 0}, sin_zero = "\000\332\303`Z'\342", <incomplete sequence \366>},
          sin6 = {sin6_family = 64234, sin6_port = 26370, sin6_flowinfo = 0, sin6_addr = {__in6_u = {
                __u6_addr8 = "\000\332\303`Z'\342\366\004\000\000\000\000\000\000", __u6_addr16 = {55808, 24771, 10074,
                  63202, 4, 0, 0, 0}, __u6_addr32 = {1623448064, 4142016346, 4, 0}}}, sin6_scope_id = 49550000}}
        dst_su = {s = {sa_family = 64234, sa_data = "\002g\000\000\000\000稁-\000\000\000"}, sin = {sin_family = 64234,
            sin_port = 26370, sin_addr = {s_addr = 0}, sin_zero = "稁-\000\000\000"}, sin6 = {sin6_family = 64234,
            sin6_port = 26370, sin6_flowinfo = 0, sin6_addr = {__in6_u = {
                __u6_addr8 = "稁-\000\000\000\000\352\372\002g\000\000\000", __u6_addr16 = {43239, 11649, 0, 0, 64234,
                  26370, 0, 0}, __u6_addr32 = {763472103, 0, 1728248554, 0}}}, sin6_scope_id = 763472544}}
        error = <optimized out>
        parallel_handling = <optimized out>
        max_chunks = <optimized out>
        __FUNCTION__ = "tcp_read_req"
#31 0x000055c13b64fe70 in handle_io (fm=<optimized out>, idx=idx@entry=2, event_type=event_type@entry=1)
    at net/net_tcp_proc.c:329
        ret = 0
        n = <optimized out>
        con = 0x7f1f20042fc0
        s = 129
        rw = <optimized out>
        resp = <optimized out>
        response = {139771657859008, 1}
        __FUNCTION__ = "handle_io"
#32 0x000055c13b650f58 in io_wait_loop_epoll (h=<optimized out>, t=<optimized out>, repeat=<optimized out>)
    at net/../io_wait_loop.h:305
        ep_event = {events = 997096623, data = {ptr = 0x879a7170000055c1, fd = 21953, u32 = 21953,
            u64 = 9771247067384010177}}
        curr_time = <optimized out>
        ret = <optimized out>
        n = <optimized out>
        r = 2
        e = <optimized out>
        fd = <optimized out>
        i = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        n = <optimized out>
        r = <optimized out>
        i = <optimized out>
        e = <optimized out>
        ep_event = <optimized out>
        fd = <optimized out>
        curr_time = <optimized out>
        again = <optimized out>
        __FUNCTION__ = <optimized out>
        error = <optimized out>
#33 tcp_worker_proc_loop () at net/net_tcp_proc.c:442
        __FUNCTION__ = "tcp_worker_proc_loop"
#34 0x000055c13b652f29 in tcp_start_processes (chd_rank=0x55c13b74bdc4 <chd_rank>, startup_done=0x7f1f20041918)
    at net/net_tcp.c:2119
        r = 0
        n = <optimized out>
        p_id = <optimized out>
        reader_fd = {123, 124}
        si = <optimized out>
        ifp_sr_tcp = {proc_desc = 0x55c13b6e4c5d "SIP receiver TCP", flags = 4, type = TYPE_TCP}
        error = <optimized out>
        __FUNCTION__ = "tcp_start_processes"
#35 0x000055c13b4b8534 in main_loop () at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/main.c:243
        startup_done = 0x7f1f20041918
        last_check = 0
        rc = <optimized out>
        chd_rank = <optimized out>
        startup_done = <optimized out>
        last_check = <optimized out>
        rc = <optimized out>
        profiling_handler = <optimized out>
        __FUNCTION__ = <optimized out>
        error = <optimized out>
#36 main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/opensips-3.4.3-14.fc39.x86_64/main.c:966
        c = <optimized out>
        r = <optimized out>
        tmp = 0x7ffe879a7dd0 ""
        tmp_len = <optimized out>
        port = <optimized out>
        proto = 0
        protos_no = <optimized out>
        options = 0x55c13b69b2e8 "f:cCm:M:b:l:n:N:rRvdDFEVhw:t:u:g:p:P:G:W:o:a:k:s:"
        ret = -1
        seed = 1318473389
        rfd = <optimized out>
        procs_no = <optimized out>
        __FUNCTION__ = "main"
(gdb)

Reproduce

resume route is misspelled. 

route[API_CALL] {
async(rest_get("http://apilayer.net/api/validate?$var(data)", $json(lookup), $var(retrive), $var(rcode)), DST_LOOKUP_RESUME);
}

route[resume] {

}
@bogdan-iancu
Copy link
Member

Could you please test this fix:

diff --git a/modules/tm/async.c b/modules/tm/async.c
index b447907d7..037d0511a 100644
--- a/modules/tm/async.c
+++ b/modules/tm/async.c
@@ -63,7 +63,8 @@ static inline void run_resume_route( struct script_route_ref * resume_route,
 {
        /* run the resume route and if it ends the msg handling (no other aysnc
         * started), run the post script callbacks. */
-       if ( (run_top_route(sroutes->request[resume_route->idx], msg) & ACT_FL_TBCONT) == 0 )
+       if ( ref_script_route_check_and_update( resume_route) &&
+       (run_top_route(sroutes->request[resume_route->idx], msg) & ACT_FL_TBCONT) == 0 )
                if (run_post_cb)
                        exec_post_req_cb(msg);
 }

@bogdan-iancu bogdan-iancu self-assigned this Oct 18, 2024
@bogdan-iancu bogdan-iancu added this to the 3.4.9 milestone Oct 18, 2024
@volga629-1
Copy link
Author

Thank you, I will test soon

@volga629-1
Copy link
Author

Hello @bogdan-iancu
I tested patch it works as intended, no more crash.
Is there are should be validation if resume route should be present in Async mode

DST_LOOKUP_RESUME2 is not exist, only DST_LOOKUP_RESUME.

Nov  3 20:10:50 dev1-msg opensips[22076]: [OSIP-250-SMPP] [22076]: DBG:core:dup_ref_script_route_in_shm: dupping 0x7f1a41a58cf8 [DST_LOOKUP_RESUME2], idx -1, ver/cnt 1, to new 0x7f1a3204a6d0 [DST_LOOKUP_RESUME2], idx -1, ver/cnt 1
Nov  3 20:10:50 dev1-msg opensips[22076]: [OSIP-250-SMPP] [22076]: ERROR:tm:t_handle_async: failed dup resume route -> act in sync mode

@volga629-1
Copy link
Author

mi reload_routes detect correctly.

 tail -f  /var/log/opensips/opensips.log
Nov  3 20:29:12 dev1-msg opensips[22070]: [OSIP-250-SMPP] [22070]: WARNING:core:update_all_script_route_refs: route [DST_LOOKUP_RESUME2] not found when updating refs

@github-actions GitHub Actions
Copy link

Any updates here? No progress has been made in the last 15 days, marking as stale. Will close this issue if no further updates are made in the next 30 days.

@github-actions github-actions bot added the stale label Nov 19, 2024
@volga629-1
Copy link
Author

In progress.

@stale stale bot removed the stale label Nov 24, 2024
@github-actions GitHub Actions
Copy link

Any updates here? No progress has been made in the last 15 days, marking as stale. Will close this issue if no further updates are made in the next 30 days.

@github-actions github-actions bot added the stale label Dec 10, 2024
@volga629-1
Copy link
Author

in progress

@stale stale bot removed the stale label Dec 10, 2024
@github-actions GitHub Actions
Copy link

Any updates here? No progress has been made in the last 15 days, marking as stale. Will close this issue if no further updates are made in the next 30 days.

@github-actions github-actions bot added the stale label Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bogdan-iancu bogdan-iancu

Labels
bug stale
Projects
None yet
Milestone
3.4.11
Development

No branches or pull requests
2 participants
@bogdan-iancu @volga629-1