Bug 241801
| Summary: | lang/python38: regression: hang in locking in multiprocessing.Pool | ||
|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Dmitry Marakasov <amdmi3> |
| Component: | Individual Port(s) | Assignee: | freebsd-python (Nobody) <python> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | CC: | kai, wen |
| Priority: | --- | Flags: | kai:
maintainer-feedback+
|
| Version: | Latest | ||
| Hardware: | Any | ||
| OS: | Any | ||
| URL: | https://bugs.python.org/issue38744 | ||
| See Also: | https://github.com/python/cpython/pull/19023 | ||
Created upstream issue: https://bugs.python.org/issue38744 (In reply to Dmitry Marakasov from comment #1) Nice work ! (In reply to Dmitry Marakasov from comment #1) This was fixed in BPO 39360 by upstream and was backported to Python 3.8.3 . Thank you for reporting it here and at upstream! Please re-open this bug if there are still issues. |
This simple program from multiprocessing import Pool from time import sleep Pool().map(sleep, [0.01] * 10) works fine with python 3.7, but is likely (about 20-50% probability on my 4 core box) to hang with python 3.8. Example backtraces after interruption: % python3.8 1.py ^CException ignored in: <Finalize object, dead> Traceback (most recent call last): File "/usr/local/lib/python3.8/multiprocessing/util.py", line 201, in __call__ Process ForkPoolWorker-2: Process ForkPoolWorker-4: res = self._callback(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 689, in _terminate_pool cls._help_stuff_finish(inqueue, task_handler, len(pool)) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 674, in _help_stuff_finish inqueue._rlock.acquire() KeyboardInterrupt: Process ForkPoolWorker-3: Process ForkPoolWorker-1: % python3.8 1.py ^CException ignored in: <Finalize object, dead> Traceback (most recent call last): File "/usr/local/lib/python3.8/multiprocessing/util.py", line 201, in __call__ Process ForkPoolWorker-3: Process ForkPoolWorker-4: Process ForkPoolWorker-1: res = self._callback(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 689, in _terminate_pool cls._help_stuff_finish(inqueue, task_handler, len(pool)) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 674, in _help_stuff_finish inqueue._rlock.acquire() KeyboardInterrupt: Process ForkPoolWorker-2: % python3.8 1.py ^CException ignored in: <Finalize object, dead> Process ForkPoolWorker-2: Process ForkPoolWorker-3: Traceback (most recent call last): File "/usr/local/lib/python3.8/multiprocessing/util.py", line 201, in __call__ Process ForkPoolWorker-1: res = self._callback(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 689, in _terminate_pool cls._help_stuff_finish(inqueue, task_handler, len(pool)) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 674, in _help_stuff_finish Traceback (most recent call last): Traceback (most recent call last): File "/usr/local/lib/python3.8/multiprocessing/process.py", line 313, in _bootstrap self.run() File "/usr/local/lib/python3.8/multiprocessing/process.py", line 313, in _bootstrap self.run() File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in worker task = get() File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in worker task = get() File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 355, in get with self._rlock: File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 355, in get with self._rlock: File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() KeyboardInterrupt KeyboardInterrupt inqueue._rlock.acquire() KeyboardInterrupt: Traceback (most recent call last): File "/usr/local/lib/python3.8/multiprocessing/process.py", line 313, in _bootstrap self.run() File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in worker task = get() File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 356, in get res = self._reader.recv_bytes() File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 216, in recv_bytes buf = self._recv_bytes(maxlength) File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes buf = self._recv(4) File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 379, in _recv chunk = read(handle, remaining) KeyboardInterrupt Process ForkPoolWorker-4: Should I submit this upstream?