pygrass: How to allow Ctrl-C to stop all subprocesses of GridModule

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

pygrass: How to allow Ctrl-C to stop all subprocesses of GridModule

Moritz Lennert
And another question on GridModule:

For the same module using GridModule to split the task of edge detection
in a raster map into many parallel tasks, when I want to stop the module
run using Ctrl-C I get tons of

Process PoolWorker-315:
Traceback (most recent call last):
Traceback (most recent call last):
   File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
_bootstrap
   File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
_bootstrap
     self.run()
     self.run()
   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
   File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
     self._target(*self._args, **self._kwargs)
     self._target(*self._args, **self._kwargs)
   File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
   File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
     task = get()
   File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
     task = get()
   File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
     racquire()
     racquire()
KeyboardInterrupt


and cannot stop the module, even when flooding the input with Ctrl-Cs.

Any hint how to allow Ctrl-C to stop all subprocesses ?

Moritz
_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: pygrass: How to allow Ctrl-C to stop all subprocesses of GridModule

NikosAlexandris
* Moritz Lennert <[hidden email]> [2018-03-08 15:05:46 +0100]:

>And another question on GridModule:
>
>For the same module using GridModule to split the task of edge
>detection in a raster map into many parallel tasks, when I want to
>stop the module run using Ctrl-C I get tons of
>
>Process PoolWorker-315:
>Traceback (most recent call last):
>Traceback (most recent call last):
>  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
>_bootstrap
>  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
>_bootstrap
>    self.run()
>    self.run()
>  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
>  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
>    self._target(*self._args, **self._kwargs)
>    self._target(*self._args, **self._kwargs)
>  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
>  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in worker
>    task = get()
>  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
>    task = get()
>  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in get
>    racquire()
>    racquire()
>KeyboardInterrupt
>
>
>and cannot stop the module, even when flooding the input with Ctrl-Cs.
>
>Any hint how to allow Ctrl-C to stop all subprocesses ?
>
>Moritz
Moritz, could it be a case related to the ones mentioned in

https://stackoverflow.com/a/1635084/1172302
https://stackoverflow.com/a/11816038/1172302

?

Nikos



_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pygrass: How to allow Ctrl-C to stop all subprocesses of GridModule

Moritz Lennert
Hi Nikos !

Am 9. März 2018 09:01:25 MEZ schrieb Nikos Alexandris <[hidden email]>:

>* Moritz Lennert <[hidden email]> [2018-03-08 15:05:46
>+0100]:
>
>>And another question on GridModule:
>>
>>For the same module using GridModule to split the task of edge
>>detection in a raster map into many parallel tasks, when I want to
>>stop the module run using Ctrl-C I get tons of
>>
>>Process PoolWorker-315:
>>Traceback (most recent call last):
>>Traceback (most recent call last):
>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
>>_bootstrap
>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
>>_bootstrap
>>    self.run()
>>    self.run()
>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in
>run
>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in
>run
>>    self._target(*self._args, **self._kwargs)
>>    self._target(*self._args, **self._kwargs)
>>  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in
>worker
>>  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in
>worker
>>    task = get()
>>  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in
>get
>>    task = get()
>>  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in
>get
>>    racquire()
>>    racquire()
>>KeyboardInterrupt
>>
>>
>>and cannot stop the module, even when flooding the input with Ctrl-Cs.
>>
>>Any hint how to allow Ctrl-C to stop all subprocesses ?
>>
>>Moritz
>
>Moritz, could it be a case related to the ones mentioned in
>
>https://stackoverflow.com/a/1635084/1172302
>https://stackoverflow.com/a/11816038/1172302



Related yes, but AFAIU GridModule uses multiprocessing, not multithreading. I think it should be possible to modify something in the GridModule class so that it interrupts, but not sure about what exactly (apparently this is specifically a problem in Python 2. In Python 3 it should happen automatically IIUC.

Moritz
_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: pygrass: How to allow Ctrl-C to stop all subprocesses of GridModule

NikosAlexandris
* Moritz Lennert <[hidden email]> [2018-03-09 09:16:01 +0100]:

>Hi Nikos !
>
>Am 9. März 2018 09:01:25 MEZ schrieb Nikos Alexandris <[hidden email]>:
>>* Moritz Lennert <[hidden email]> [2018-03-08 15:05:46
>>+0100]:
>>
>>>And another question on GridModule:
>>>
>>>For the same module using GridModule to split the task of edge
>>>detection in a raster map into many parallel tasks, when I want to
>>>stop the module run using Ctrl-C I get tons of
>>>
>>>Process PoolWorker-315:
>>>Traceback (most recent call last):
>>>Traceback (most recent call last):
>>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
>>>_bootstrap
>>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in
>>>_bootstrap
>>>    self.run()
>>>    self.run()
>>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in
>>run
>>>  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in
>>run
>>>    self._target(*self._args, **self._kwargs)
>>>    self._target(*self._args, **self._kwargs)
>>>  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in
>>worker
>>>  File "/usr/lib/python2.7/multiprocessing/pool.py", line 102, in
>>worker
>>>    task = get()
>>>  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in
>>get
>>>    task = get()
>>>  File "/usr/lib/python2.7/multiprocessing/queues.py", line 374, in
>>get
>>>    racquire()
>>>    racquire()
>>>KeyboardInterrupt
>>>
>>>
>>>and cannot stop the module, even when flooding the input with Ctrl-Cs.
>>>
>>>Any hint how to allow Ctrl-C to stop all subprocesses ?
>>>
>>>Moritz
>>
>>Moritz, could it be a case related to the ones mentioned in
>>
>>https://stackoverflow.com/a/1635084/1172302
>>https://stackoverflow.com/a/11816038/1172302
>
>
>
>Related yes, but AFAIU GridModule uses multiprocessing, not multithreading. I think it should be possible to modify something in the GridModule class so that it interrupts, but not sure about what exactly (apparently this is specifically a problem in Python 2. In Python 3 it should happen automatically IIUC.
Right. This https://stackoverflow.com/a/35134329/1172302 should be,
however, directly related. However, not portable, as per a comment.

How about this one: https://stackoverflow.com/a/44869451/1172302 (July
2017)? See
also
http://jessenoller.com/blog/2009/01/08/multiprocessingpool-and-keyboardinterrupt.

Nikos

_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pygrass: How to allow Ctrl-C to stop all subprocesses of GridModule

Glynn Clements
In reply to this post by Moritz Lennert

Moritz Lennert wrote:

> Any hint how to allow Ctrl-C to stop all subprocesses ?

If you can get their PIDs, you should be able to use os.setpgid() to
move them into the same process group as the main script. Ctrl-C (by
default) sends SIGINT to all processes in the foreground process
group.

--
Glynn Clements <[hidden email]>
_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev