Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash in macOS when using wasmtime in a Python extension #10099

Open
plusvic opened this issue Jan 24, 2025 · 6 comments
Open

Crash in macOS when using wasmtime in a Python extension #10099

plusvic opened this issue Jan 24, 2025 · 6 comments
Labels
bug Incorrect behavior in the current implementation that needs fixing

Comments

@plusvic
Copy link

plusvic commented Jan 24, 2025

This is a very esoteric issue related to the use of Mach ports for exception handling when wasmtime is running inside a Python 3.9 process. I'm the author of a multi-platform Rust library that embeds wasmtime for running WASM code, the library supports Linux, Windows and macOS. This library also have Python bindings that allow using the library from Python.

A user of the library reported a crash that was happening in a very specific environment: macOS 15.2 (both Intel and ARM are affected, and probably other macOS versions too), with Python 3.9.6 (the version that comes pre-installed in macOS). The process was being SIGKILLed when executing code deep inside wasmtime, the same code worked fine in other platforms, or in the same macOS machine with Python 3.11.

After some investigation everything pointed to the Mach ports exception handling mechanism when wasmtime is run inside Python 3.9. In fact, if I turn off the use of Mach ports in wasmtime everything works fine.

I don't have a clue on why this is happening only with Python 3.9. My guess is that it is related to Python's exception handling mechanism that somewhat is interfering with wasmtime's exception handling, and perhaps something changed in later Python versions. It could be a Python's bug that was fixed, it could a design change. For the time being I can turn off Mach ports as a workaround, but I would like to understand what's happening here.

wasmtime-python.zip

Test Case

As my library is a quite complicated project, I've created a simple Rust project that reproduces the issue (attached to this issue). It consists in a Python extension written in Rust using PyO3.

Steps to Reproduce

  • Decompress the Rust project attached to this issue
  • Build the python extension using maturin: maturin build --release
  • Install the Python extension: /usr/bin/python3 -m pip install <path to the .whl file produced by maturin>
  • Run the crash.py file included in the project: /usr/bin/python3 crash.py

Expected Results

The process should finish without crashes.

Actual Results

The process gets killed:

/usr/bin/python3 crash.py                                                                                                                    ✔
[1]    34279 killed     /usr/bin/python3 crash.py

The crash log is included below, notice that the exception type is EXC_GUARD and the code is GUARD_TYPE_MACH_PORT. According to Apple's documentation, the process is killed with EXC_GUARD when it detects a violation in some guarded resourced, usually a file descriptor or Mach port.

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               Python [34279]
Path:                  /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
Identifier:            com.apple.python3
Version:               3.9.6 (3.9.6)
Build Info:            python3-141000000000000~2583
Code Type:             X86-64 (Native)
Parent Process:        zsh [1143]
Responsible:           iTerm2 [911]
User ID:               178948

Date/Time:             2025-01-24 11:53:27.1988 +0100
OS Version:            macOS 15.2 (24C101)
Report Version:        12
Bridge OS Version:     9.2 (22P2093)
Anonymous UUID:        5FA73C73-360A-C399-E0CC-3B9EAB5492D0

Sleep/Wake UUID:       CA017020-8C6C-41A1-AE24-D06480AE16DB

Time Awake Since Boot: 80000 seconds
Time Since Wake:       5718 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_GUARD (SIGKILL)
Exception Codes:       GUARD_TYPE_MACH_PORT
Exception Codes:       0x0000000000000000, 0x0000000000000000

Termination Reason:    Namespace GUARD, Code 2305843034983497728 

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff80b5daf0e mach_msg2_trap + 10
1   libsystem_kernel.dylib        	    0x7ff80b5e95fa mach_msg2_internal + 84
2   libsystem_kernel.dylib        	    0x7ff80b5dfffb thread_set_exception_ports + 134
3   wasmtime_python.abi3.so       	       0x10d421005 wasmtime::runtime::vm::sys::unix::machports::lazy_per_thread_init::h995e1f3306bad314 + 69
4   wasmtime_python.abi3.so       	       0x10ce54cd5 wasmtime::runtime::vm::traphandlers::tls::raw::replace::h7b1ea496b7d61dcd + 85
5   wasmtime_python.abi3.so       	       0x10cd6cc95 wasmtime::runtime::vm::traphandlers::catch_traps::haad6173518c65209 + 213
6   wasmtime_python.abi3.so       	       0x10cd66c62 wasmtime::runtime::func::typed::TypedFunc$LT$Params$C$Results$GT$::call::h4ad4a8810ccdb7a1 + 354
7   wasmtime_python.abi3.so       	       0x10cd6e575 wasmtime_python::_$LT$impl$u20$wasmtime_python..sum..MakeDef$GT$::_PYO3_DEF::trampoline::hfe78f2643f499858 + 1077
8   Python3                       	       0x10c0a6ae8 0x10c024000 + 535272
9   Python3                       	       0x10c066830 _PyObject_MakeTpCall + 384
10  Python3                       	       0x10c143850 0x10c024000 + 1177680
11  Python3                       	       0x10c140557 _PyEval_EvalFrameDefault + 23527
12  Python3                       	       0x10c144693 0x10c024000 + 1181331
13  Python3                       	       0x10c13a871 PyEval_EvalCode + 81
14  Python3                       	       0x10c181ea1 0x10c024000 + 1433249
15  Python3                       	       0x10c182013 0x10c024000 + 1433619
16  Python3                       	       0x10c1802a4 PyRun_SimpleFileExFlags + 708
17  Python3                       	       0x10c19d6f0 Py_RunMain + 1840
18  Python3                       	       0x10c19db80 0x10c024000 + 1547136
19  Python3                       	       0x10c19dbdb Py_BytesMain + 43
20  dyld                          	    0x7ff80b28d2cd start + 1805

Thread 1:
0   libsystem_kernel.dylib        	    0x7ff80b5daf0e mach_msg2_trap + 10
1   libsystem_kernel.dylib        	    0x7ff80b5e95fa mach_msg2_internal + 84
2   libsystem_kernel.dylib        	    0x7ff80b5e2016 mach_msg_overwrite + 649
3   libsystem_kernel.dylib        	    0x7ff80b5db1ff mach_msg + 19
4   wasmtime_python.abi3.so       	       0x10ce55ef5 wasmtime::runtime::vm::sys::unix::machports::handler_thread::hbfc195b7bc426a10 + 277
5   wasmtime_python.abi3.so       	       0x10ce08da9 std::sys::backtrace::__rust_begin_short_backtrace::h9512e42a7b3024a7 + 9
6   wasmtime_python.abi3.so       	       0x10cdec98a core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::hd93b48b54e95ca2f + 138
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 2:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 3:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 4:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 5:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 6:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 7:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 8:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 9:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 10:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 11:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 12:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15

Thread 13:
0   libsystem_kernel.dylib        	    0x7ff80b5ddaaa __psynch_cvwait + 10
1   libsystem_pthread.dylib       	    0x7ff80b61c7a8 _pthread_cond_wait + 1193
2   wasmtime_python.abi3.so       	       0x10d44518f rayon_core::sleep::Sleep::sleep::h0f8e31d6649cc8dc + 511
3   wasmtime_python.abi3.so       	       0x10d444b54 rayon_core::registry::WorkerThread::wait_until_cold::h5e480e7580c0b42b + 276
4   wasmtime_python.abi3.so       	       0x10d1e5d14 rayon_core::registry::ThreadBuilder::run::h1666e03950a6d606 + 340
5   wasmtime_python.abi3.so       	       0x10d1ebbf2 std::sys::backtrace::__rust_begin_short_backtrace::h84248f773d70d50b + 34
6   wasmtime_python.abi3.so       	       0x10d1ea2e5 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h2bcc168af3d264f4 + 373
7   wasmtime_python.abi3.so       	       0x10d3f5c5b std::sys::pal::unix::thread::Thread::new::thread_start::hcfbe36d5799581a6 + 43
8   libsystem_pthread.dylib       	    0x7ff80b61c253 _pthread_start + 99
9   libsystem_pthread.dylib       	    0x7ff80b617bef thread_start + 15


Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x00007ff7b4470d34  rcx: 0x00007ff7b4470ca8  rdx: 0x0000003c80001513
  rdi: 0x00007ff7b4470d34  rsi: 0x0000000200000003  rbp: 0x00007ff7b4470d10  rsp: 0x00007ff7b4470ca8
   r8: 0x00000e1d00000000   r9: 0x0000080300000001  r10: 0x0000080300000103  r11: 0x0000000000000206
  r12: 0x0000000000000000  r13: 0x0000000200000003  r14: 0x000000000000002c  r15: 0x0000080300000001
  rip: 0x00007ff80b5daf0e  rfl: 0x0000000000000206  cr2: 0x0000000000000000
  
Logical CPU:     0
Error Code:      0x0100002f 
Trap Number:     133


Binary Images:
       0x10ba8d000 -        0x10ba8efff com.apple.python3 (3.9.6) <c22a284e-5ae7-3332-95f5-de456ff5188e> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
       0x10c024000 -        0x10c283fff com.apple.python3 (3.9.6) <daafc950-10a8-3a70-966c-4517ea16c3c8> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Python3
       0x10bce9000 -        0x10bcecfff _heapq.cpython-39-darwin.so (*) <8e398855-3ad0-3316-b757-e6d055a773bc> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/lib-dynload/_heapq.cpython-39-darwin.so
       0x10cd5d000 -        0x10d535fff wasmtime_python.abi3.so (*) <d05ed47c-6671-3f66-8d09-ad0a9483c278> /Users/USER/Library/Python/3.9/lib/python/site-packages/wasmtime_python/wasmtime_python.abi3.so
    0x7ff80b5da000 -     0x7ff80b615fff libsystem_kernel.dylib (*) <907f6ba6-6b4b-39d3-ba4b-d1f4244b50bc> /usr/lib/system/libsystem_kernel.dylib
    0x7ff80b287000 -     0x7ff80b312c7f dyld (*) <7e2d5f84-7d4f-3322-83cb-dd7bd247f06a> /usr/lib/dyld
               0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
    0x7ff80b616000 -     0x7ff80b621fff libsystem_pthread.dylib (*) <7324bb8c-37ad-3a44-becd-969f31077819> /usr/lib/system/libsystem_pthread.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 2
    thread_create: 0
    thread_set_state: 14

VM Region Summary:
ReadOnly portion of Libraries: Total=420.1M resident=0K(0%) swapped_out_or_unallocated=420.1M(100%)
Writable regions: Total=657.2M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=657.2M(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                           611.6M       34 
MALLOC guard page                   16K        4 
MALLOC_LARGE (reserved)            384K        1         reserved VM address space (unallocated)
STACK GUARD                         56K       14 
Stack                             42.1M       14 
VM_ALLOCATE                       2576K       13 
VM_ALLOCATE (reserved)             512K        1         reserved VM address space (unallocated)
__DATA                            4458K      276 
__DATA_CONST                      21.3M      296 
__DATA_DIRTY                       436K       93 
__FONT_DATA                        2352        1 
__LINKEDIT                       186.4M        6 
__OBJC_RO                         76.9M        1 
__OBJC_RW                         2373K        2 
__TEXT                           233.7M      303 
__TPRO_CONST                       272K        2 
shared memory                       32K        4 
===========                     =======  ======= 
TOTAL                              1.2G     1066 
TOTAL, minus reserved VM space     1.2G     1066 

Versions and Environment

Wasmtime version or commit: 29.0.1

Operating system: macOS 15.2

Architecture: both Intel and ARM

Extra Info

By disabling the use of Mach ports for exception handling (with macos_use_mach_ports) the problem disappears.

@plusvic plusvic added the bug Incorrect behavior in the current implementation that needs fixing label Jan 24, 2025
@plusvic
Copy link
Author

plusvic commented Jan 24, 2025

I added the bug label and can't remove it now. This is may not be a wasmtime bug.

@bjorn3
Copy link
Contributor

bjorn3 commented Jan 24, 2025

The crash happens when thread_set_exception_ports is called by Wasmtime:

let kret = thread_set_exception_ports(

with Python 3.9.6 (the version that comes pre-installed in macOS)

Do you happen to know the exact set of entitlements this executable was compiled with? Maybe one of the hardening entitlements prevents using exception ports?

@plusvic
Copy link
Author

plusvic commented Jan 24, 2025

The crash happens when thread_set_exception_ports is called by Wasmtime:

wasmtime/crates/wasmtime/src/runtime/vm/sys/unix/machports.rs

Line 483 in 887e5c9

let kret = thread_set_exception_ports(

with Python 3.9.6 (the version that comes pre-installed in macOS)

Do you happen to know the exact set of entitlements this executable was compiled with? Maybe one of the hardening entitlements prevents using exception ports?

The Python interpreter executable doesn't seem to specify any entitlement:

codesign -d --entitlements - /usr/bin/python3
Executable=/usr/bin/python3

@plusvic
Copy link
Author

plusvic commented Jan 24, 2025

The crash happens when thread_set_exception_ports is called by Wasmtime:
wasmtime/crates/wasmtime/src/runtime/vm/sys/unix/machports.rs
Line 483 in 887e5c9
let kret = thread_set_exception_ports(

with Python 3.9.6 (the version that comes pre-installed in macOS)

Do you happen to know the exact set of entitlements this executable was compiled with? Maybe one of the hardening entitlements prevents using exception ports?

The Python interpreter executable doesn't seem to specify any entitlement:

codesign -d --entitlements - /usr/bin/python3
Executable=/usr/bin/python3

With this other path I get a different result, but nothing that prevents the use of exception ports.

codesign -d --entitlements - /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
Executable=/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
[Dict]
	[Key] com.apple.application-identifier
	[Value]
		[String] com.apple.python3

@alexcrichton
Copy link
Member

Thanks for the thorough report and reproduction steps! I took a look at this but alas don't have much to add. It deifnitely seems like macOS doesn't like the call to thread_set_exception_ports but I'm not really sure why. Searching around isn't turning up much that seems relevant, and I'm definitely no macOS expert myself to know what's going on here...

@plusvic
Copy link
Author

plusvic commented Jan 27, 2025

To make this issue even more strange, everything is OK with a Python binary downloaded from python.org. Exactly the same version (Python 3.9.6), it crashes with the Python binary provided by Apple, but works correctly with the one from python.org.

It looks like Apple is making some modifications to the Python binary they distribute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior in the current implementation that needs fixing
Projects
None yet
Development

No branches or pull requests

3 participants