more interesting failure for reading fp_range of a NULL map

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

more interesting failure for reading fp_range of a NULL map

ychemin
Hi,

dealing with large amount of maps, some of them process to NULL on all pixels.
when accessing fp_range in cell_misc, the file is empty, thus returning such an error:

WARNING: Unable to read fp range file for <NULL_map_test@PERMANENT>

While I understand this is undefined because of all pixels being NULL,
would it be possible to have a more interesting output?

This could help in screening NULL maps too...

Thank you,
Yann

--
Yann Chemin
Researcher@IWMI
Skype/FB: yann.chemin


_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

Glynn Clements

Yann Chemin wrote:

> dealing with large amount of maps, some of them process to NULL on all
> pixels.
> when accessing fp_range in cell_misc, the file is empty, thus returning
> such an error:
>
> WARNING: Unable to read fp range file for <NULL_map_test@PERMANENT>
>
> While I understand this is undefined because of all pixels being NULL,
> would it be possible to have a more interesting output?

lib/raster/range.c is inconsistent in this regard. When writing an FP
range, it writes an empty file if the range is null:

    /* if range hasn't been updated, write empty file meaning Nulls */

When reading an empty f_range file, it generates the above warning
(the FPRange structure is initialised with min and max both null).

It would be a simple matter to change the warning (e.g. "range is
null") or remove it. A warning might be useful, as I can imagine
modules failing in interesting ways if an input map is all-null.

--
Glynn Clements <[hidden email]>
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

ychemin



On 6 March 2013 23:35, Glynn Clements <[hidden email]> wrote:

Yann Chemin wrote:

> dealing with large amount of maps, some of them process to NULL on all
> pixels.
> when accessing fp_range in cell_misc, the file is empty, thus returning
> such an error:
>
> WARNING: Unable to read fp range file for <NULL_map_test@PERMANENT>
>
> While I understand this is undefined because of all pixels being NULL,
> would it be possible to have a more interesting output?

lib/raster/range.c is inconsistent in this regard. When writing an FP
range, it writes an empty file if the range is null:

    /* if range hasn't been updated, write empty file meaning Nulls */

When reading an empty f_range file, it generates the above warning
(the FPRange structure is initialised with min and max both null).

It would be a simple matter to change the warning (e.g. "range is
null") or remove it. A warning might be useful, as I can imagine
modules failing in interesting ways if an input map is all-null.


In the same way as there is a pixel based function to assess a NULL pixel value,
it would be very interesting to have a function to test an all NULL raster map status.

Time series analysis would be the first to benefit.
 
--
Glynn Clements <[hidden email]>



--
Yann Chemin
Researcher@IWMI
Skype/FB: yann.chemin


_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

hamish-2
Yann:
> In the same way as there is a pixel based function to assess a
> NULL pixel value, it would be very interesting to have a
> function to test an all NULL raster map status.

see 'r.info -r' for overall map, or r.univar for within-region window.


?
Hamish
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

ychemin
Hi Hamish,

r.info
------
r.info -r map=nullmap@PERMANENT                                      
min=-nan
max=-nan
WARNING: Unable to read fp range file for <nullmap@PERMANENT>
(Thu Mar  7 13:15:05 2013) Command finished (0 sec)                       

That is a module level way, i was wondering about a C function like the one below:

/*A fatal error when NULL map*/
if(Rast_is_null_map(mapname, mapset)<0){
   G_fatal("Input file has only null values");
} else {
  /* do some useful thing */
}


/*A discarding map error when NULL map*/
if(Rast_is_null_map(mapname, mapset)<0){
   G_warning("Input file %s has only null values", mapname);
   /*do not load that map*/
   /*remove that map from the list of temporal iteration*/
} else {
  /* do some useful thing */
}





On 7 March 2013 13:12, Hamish <[hidden email]> wrote:
Yann:
> In the same way as there is a pixel based function to assess a
> NULL pixel value, it would be very interesting to have a
> function to test an all NULL raster map status.

see 'r.info -r' for overall map, or r.univar for within-region window.


?
Hamish



--
Yann Chemin
Researcher@IWMI
Skype/FB: yann.chemin


_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

hamish-2
In reply to this post by ychemin
Yann wrote:
> r.info
> ------
> r.info -r  map=nullmap@PERMANENT
> min=-nan
> max=-nan
> WARNING: Unable to read fp range file for <nullmap@PERMANENT>

just to report that the warning does not show up in devbr6;
r.univar reports no output besides 100% (it's a bug which comes
from the multi-map support, where it skips over empty maps but
doesn't warn you when none of the input maps contain data);
and the f_range file in cell_misc/ for it is 0 bytes.


Hamish
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

Glynn Clements
In reply to this post by ychemin

Yann Chemin wrote:

> That is a module level way, i was wondering about a C function like the one
> below:
>
> /*A fatal error when NULL map*/
> if(Rast_is_null_map(mapname, mapset)<0){
>    G_fatal("Input file has only null values");
> } else {
>   /* do some useful thing */
> }

1. In general, there's nothing erroneous about an all-null input map.
Most modules should treat such a map the same as any other map.

2. Whether or not a map contains any non-null cells is usually
irrelevant. What matters is whether there are any non-null cells
within the current region (and after any mask has been applied). And
this can't be determined without actually reading the map.

So given that the map has to be read anyhow, the module should simply
operate normally, keeping track of whether any non-null values have
been read so that any final calculation doesn't make assumptions about
e.g. the count of non-null cells being non-zero, having a valid
minimum and maximum, etc.

If, for a specific module, all-null inputs are common, it might be
worth checking whether the range has null minimum and/or maximum
(they'll be either both null or both non-null) as an optimisation. But
I would expect this to be the exception rather than the rule.

--
Glynn Clements <[hidden email]>
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

Markus Neteler
(back to an older yet new issue)


On Thu, Mar 7, 2013 at 10:11 PM, Glynn Clements
<[hidden email]> wrote:

>
> Yann Chemin wrote:
>
>> That is a module level way, i was wondering about a C function like the one
>> below:
>>
>> /*A fatal error when NULL map*/
>> if(Rast_is_null_map(mapname, mapset)<0){
>>    G_fatal("Input file has only null values");
>> } else {
>>   /* do some useful thing */
>> }
>
> 1. In general, there's nothing erroneous about an all-null input map.
> Most modules should treat such a map the same as any other map.
>
> 2. Whether or not a map contains any non-null cells is usually
> irrelevant. What matters is whether there are any non-null cells
> within the current region (and after any mask has been applied). And
> this can't be determined without actually reading the map.
>
> So given that the map has to be read anyhow, the module should simply
> operate normally, keeping track of whether any non-null values have
> been read so that any final calculation doesn't make assumptions about
> e.g. the count of non-null cells being non-zero, having a valid
> minimum and maximum, etc.
>
> If, for a specific module, all-null inputs are common, it might be
> worth checking whether the range has null minimum and/or maximum
> (they'll be either both null or both non-null) as an optimisation. But
> I would expect this to be the exception rather than the rule.


On G+ a user brought up this issue:

On Tue, Sep 30, 2014 at 11:20 AM, xxx wrote:
>
> r.watershed generates this error:
>
> WARNING: Unable to read fp range file for <accumulation@Utente_1>

I would vote for an improvement of the error message since a common
user will not know what to do next.

thanks
Markus
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Reply | Threaded
Open this post in threaded view
|

Re: more interesting failure for reading fp_range of a NULL map

Markus Neteler
On Wed, Oct 1, 2014 at 4:51 PM, Markus Neteler <[hidden email]> wrote:
> On Thu, Mar 7, 2013 at 10:11 PM, Glynn Clements
> On Tue, Sep 30, 2014 at 11:20 AM, xxx wrote:
>>
>> r.watershed generates this error:
>>
>> WARNING: Unable to read fp range file for <accumulation@Utente_1>
>
> I would vote for an improvement of the error message since a common
> user will not know what to do next.

FYI
Markus Metz has addressed this in r62142 and r62143.

Markus
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev