[GRASS GIS] #3857: r.mapcalc unable to rename null and cell files

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[GRASS GIS] #3857: r.mapcalc unable to rename null and cell files

GRASS GIS
#3857: r.mapcalc unable to rename null and cell files
-------------------------+-------------------------
 Reporter:  Hygsson      |      Owner:  grass-dev@…
     Type:  defect       |     Status:  new
 Priority:  major        |  Milestone:  7.6.2
Component:  Raster       |    Version:  unspecified
 Keywords:  r.mapcalc    |        CPU:  x86-64
 Platform:  MSWindows 7  |
-------------------------+-------------------------
 **Note:** This works on Ubuntu without any problems.

 While using GRASS GIS 7.6.1 on 64-bit Windows 7, I encountered a problem
 with r.mapcalc. For a simple stochastic simulation of watershed, I used a
 following script (here it is recreated for Spearfish dataset).

 {{{
 g.region raster=elevation.dem@PERMANENT
 r.surf.random --overwrite output=frequency min=0 max=0
 for (( n=1 ; n<21 ; n=$n+1 ))
 do
         r.surf.random --overwrite output=errors min=-15 max=15
         r.mapcalc "terrain = elevation.dem@PERMANENT + errors@PERMANENT"
 --overwrite
         r.watershed --overwrite elevation=terrain@PERMANENT
 accumulation=accum$n
         r.reclass --overwrite input=accum$n@PERMANENT output=flow$n
 rules=C:/Users/Hygsson/Desktop/test/reclass.rules
         r.mapcalc "frequency = flow$n + frequency" --overwrite
 done
 }}}

 reclass.rules file has a following content:

 {{{
 -500 thru 654 = 0
 * = 1
 }}}

 I tried to do it step by step without using the script. **The problem
 arises with this step:**

 {{{
 r.mapcalc "frequency = flow$n + frequency" --overwrite
 }}}

 I get this output for each iteration of the loop. Only numbers behind
 'unknown/' are different.

 {{{
 WARNING: Unable to rename null file
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\.tmp/unknown\22368.1'
             to
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\cell_misc\frequency\nullcmpr':
             File exists
 WARNING: Unable to rename cell file
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\.tmp/unknown\22368.0'
             to
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\fcell\frequency':
             File exists
 }}}

 The desired 'frequency' raster is created, but obviously in a wrong
 manner, having only one value.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3857>
GRASS GIS <https://grass.osgeo.org>


_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: [GRASS GIS] #3857: r.mapcalc unable to rename null and cell files

GRASS GIS
#3857: r.mapcalc unable to rename null and cell files
----------------------+-------------------------
  Reporter:  Hygsson  |      Owner:  grass-dev@…
      Type:  defect   |     Status:  new
  Priority:  major    |  Milestone:  7.6.2
 Component:  Raster   |    Version:  unspecified
Resolution:           |   Keywords:  r.mapcalc
       CPU:  x86-64   |   Platform:  MSWindows 7
----------------------+-------------------------

Comment (by wenzeslaus):

 Overwriting map you are reading from is not allowed according to the
 documentation:

 //A map cannot be used both as an input and as an output as in this
 invalid expression `oldmap = oldmap + 1`, instead a subsequent rename
 using `g.rename` is needed when the same name is desired: //

 {{{
 r.mapcalc "newmap = oldmap + 1"
 g.rename raster=newmap,oldmap
 }}}

 https://grass.osgeo.org/grass76/manuals/r.mapcalc.html#using-the-same-map-
 for-input-and-output-results

 It may work in some cases, but it is not guaranteed. Maybe a check of the
 inputs and outputs and an error message on Windows and a warning on other
 platforms would be backward-compatible solution. Would you mind opening a
 new ticket? On the other hand, having this feature (`a = a + 1`) seems
 reasonable (but supporting it is more complex).

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3857#comment:1>
GRASS GIS <https://grass.osgeo.org>


_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: [GRASS GIS] #3857: r.mapcalc unable to rename null and cell files

GRASS GIS
In reply to this post by GRASS GIS
#3857: r.mapcalc unable to rename null and cell files
----------------------+-------------------------
  Reporter:  Hygsson  |      Owner:  grass-dev@…
      Type:  defect   |     Status:  new
  Priority:  major    |  Milestone:  7.6.2
 Component:  Raster   |    Version:  unspecified
Resolution:           |   Keywords:  r.mapcalc
       CPU:  x86-64   |   Platform:  MSWindows 7
----------------------+-------------------------

Comment (by hellik):

 Replying to [ticket:3857 Hygsson]:
 > **Note:** This works on Ubuntu without any problems.
 >
 > While using GRASS GIS 7.6.1 on 64-bit Windows 7, I encountered a problem
 with r.mapcalc. For a simple stochastic simulation of watershed, I used a
 following script (here it is recreated for Spearfish dataset).
 >
 > {{{
 > g.region raster=elevation.dem@PERMANENT
 > r.surf.random --overwrite output=frequency min=0 max=0
 > for (( n=1 ; n<21 ; n=$n+1 ))
 > do
 >       r.surf.random --overwrite output=errors min=-15 max=15
 >       r.mapcalc "terrain = elevation.dem@PERMANENT + errors@PERMANENT"
 --overwrite
 >       r.watershed --overwrite elevation=terrain@PERMANENT
 accumulation=accum$n
 >       r.reclass --overwrite input=accum$n@PERMANENT output=flow$n
 rules=C:/Users/Hygsson/Desktop/test/reclass.rules
 >       r.mapcalc "frequency = flow$n + frequency" --overwrite
 > done
 > }}}
 >
 > reclass.rules file has a following content:
 >
 > {{{
 > -500 thru 654 = 0
 > * = 1
 > }}}
 >
 > I tried to do it step by step without using the script. **The problem
 arises with this step:**
 >
 > {{{
 > r.mapcalc "frequency = flow$n + frequency" --overwrite
 > }}}
 >
 > I get this output for each iteration of the loop. Only numbers behind
 'unknown/' are different.
 >
 > {{{
 > WARNING: Unable to rename null file
 >
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\.tmp/unknown\22368.1'
 >             to
 >
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\cell_misc\frequency\nullcmpr':
 >             File exists
 > WARNING: Unable to rename cell file
 >
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\.tmp/unknown\22368.0'
 >             to
 >
 'C:\Users\Hygsson\Documents\grassdata\Spearfish60_grass7\PERMANENT\fcell\frequency':
 >             File exists
 > }}}
 >
 > The desired 'frequency' raster is created, but obviously in a wrong
 manner, having only one value.


 {{{
 r.mapcalc "frequency = flow$n + frequency" --overwrite
 }}}

 it is an operating system issue/feature/limitation of MS windows, that you
 are not able to read and write the same file at the same time.

 therefore it's not a winGRASS bug.

 as Vaclav already mentioned, it may work in linux, but it's not garanteed
 that it always work in every linux.

 so I tend to a ''won't fix'' of the ticket.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3857#comment:2>
GRASS GIS <https://grass.osgeo.org>


_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: [GRASS GIS] #3857: r.mapcalc unable to rename null and cell files

GRASS GIS
In reply to this post by GRASS GIS
#3857: r.mapcalc unable to rename null and cell files
----------------------+-------------------------
  Reporter:  Hygsson  |      Owner:  grass-dev@…
      Type:  defect   |     Status:  closed
  Priority:  major    |  Milestone:  7.6.2
 Component:  Raster   |    Version:  unspecified
Resolution:  wontfix  |   Keywords:  r.mapcalc
       CPU:  x86-64   |   Platform:  MSWindows 7
----------------------+-------------------------
Changes (by Hygsson):

 * status:  new => closed
 * resolution:   => wontfix


Comment:

 OK, thank you for your replies. It's good to know that the issue is on
 Windows' side. I am therefore closing this ticket and will open a new one
 regarding that input/output check suggested by Wenzeslaus.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3857#comment:3>
GRASS GIS <https://grass.osgeo.org>


_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: [GRASS GIS] #3857: r.mapcalc unable to rename null and cell files

GRASS GIS
In reply to this post by GRASS GIS
#3857: r.mapcalc unable to rename null and cell files
----------------------+-------------------------
  Reporter:  Hygsson  |      Owner:  grass-dev@…
      Type:  defect   |     Status:  closed
  Priority:  major    |  Milestone:  7.6.2
 Component:  Raster   |    Version:  unspecified
Resolution:  wontfix  |   Keywords:  r.mapcalc
       CPU:  x86-64   |   Platform:  MSWindows 7
----------------------+-------------------------

Comment (by glynn):

 Replying to [comment:2 hellik]:

 > it is an operating system issue/feature/limitation of MS windows, that
 you are not able to read and write the same file at the same time.

 By itself, that wouldn't be an issue. GRASS raster output doesn't modify
 files in-place, but creates a temporary file which is renamed into place
 by Rast_close(). But r.mapcalc doesn't close the input maps, so their
 cell/fcell/null files are still open when the output maps are closed. A
 fix **might** be as simple as calling close_maps() from
 [source:grass/trunk/raster/r.mapcalc/evaluate.c#L396 execute()], before
 the output maps are closed.

 The temporary file is used for cell, fcell and null files. Other files are
 modified in-place, but these are normally either closed before raster I/O
 starts or opened after raster I/O has completed.

 Aside: this is why G_tempfile() creates files in the mapset, not /tmp or
 $TMPDIR; they have to be on the same filesystem as the mapset so that
 rename() works.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3857#comment:4>
GRASS GIS <https://grass.osgeo.org>


_______________________________________________
grass-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-dev