get the shifted minimum from a raster series

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

get the shifted minimum from a raster series

FrankD
Hello,

I try to find how to get the shifted minimum value of a series of raster
map. I want to shift by one, or more, the "minimum" value found on each
cell to get the "almost minimum" of the series. Is there an available
function/method to do that ?

Thank you for your help

Frank
_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

Ken Mankoff
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

Veronica Andreo
Hi Ken,

There isn't any build-in function for that in grass, afaik. Depending on how long is your series, you could shift the map list you use in each run and then use r.univar on each output from r.series to get the minimum and by comparison, get the second minimum of the series of maps (a scalar). 

However, if what you need is a map of second minimums per pixel, I believe that might require some programming for a new function... Do you know any other software which has this function? Maybe, if there's such thing in Python for example, it could be recycled or used with grass maps in a script... Dunno, just thinking out loud

best,
Vero

El jue., 7 feb. 2019 07:16, Ken Mankoff <[hidden email]> escribió:
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

FrankD

Hi,

Thanks for your reply. I can have numerous raster (several hundreds). I want to find a minimum but with a tolerance, it's why I want to be able to shift the value. In fact I want to sort the value of my all input raster for each x,y cell and get the second or third, or any position in the sorted list, to build my output raster. My application is to get the area not visible from a road (I did 1600 viewshed raster from the roads every 250m). I want a tolerance in order that one point of view (top of hill) does not reduce too much the hidden area. To do so, a fixed number of my point of view (20%) does not reduce the hidden area. The result is the area is hidden from 80% of the road.

I already wrote a python script that read all rasters for each x,y cell and make a array(), sort the array values, and build a raster. It works on reasonable number of cell, but seems to fails with large number of cells. I'm not a programmer !

I wondered if this kind of treatment exists already in Grass.

If somebody wants to hemps me to develop this, I could be nice. But I think it could be an improvement of r.series function.

Sorry for my bad english, I hope you have understand my wishes !

Cheers

Frank

Le 07/02/2019 à 12:04, Veronica Andreo a écrit :
Hi Ken,

There isn't any build-in function for that in grass, afaik. Depending on how long is your series, you could shift the map list you use in each run and then use r.univar on each output from r.series to get the minimum and by comparison, get the second minimum of the series of maps (a scalar). 

However, if what you need is a map of second minimums per pixel, I believe that might require some programming for a new function... Do you know any other software which has this function? Maybe, if there's such thing in Python for example, it could be recycled or used with grass maps in a script... Dunno, just thinking out loud

best,
Vero

El jue., 7 feb. 2019 07:16, Ken Mankoff <[hidden email]> escribió:
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

FrankD

I forgot one thing to well understand my wish. Each viewshed raster is a r.series (250-sum) of 250 r.viewshed with 2m target and 0 upto 250m terrain height. So each raster gives the hidden height of a building (250m max) anywhere on the region from a specific point of view. The purpose is to get the area of hidden wind turbine from roads for a specific wind turbine height.

Cheers,

Frank

Le 07/02/2019 à 12:48, Frank David a écrit :

Hi,

Thanks for your reply. I can have numerous raster (several hundreds). I want to find a minimum but with a tolerance, it's why I want to be able to shift the value. In fact I want to sort the value of my all input raster for each x,y cell and get the second or third, or any position in the sorted list, to build my output raster. My application is to get the area not visible from a road (I did 1600 viewshed raster from the roads every 250m). I want a tolerance in order that one point of view (top of hill) does not reduce too much the hidden area. To do so, a fixed number of my point of view (20%) does not reduce the hidden area. The result is the area is hidden from 80% of the road.

I already wrote a python script that read all rasters for each x,y cell and make a array(), sort the array values, and build a raster. It works on reasonable number of cell, but seems to fails with large number of cells. I'm not a programmer !

I wondered if this kind of treatment exists already in Grass.

If somebody wants to hemps me to develop this, I could be nice. But I think it could be an improvement of r.series function.

Sorry for my bad english, I hope you have understand my wishes !

Cheers

Frank

Le 07/02/2019 à 12:04, Veronica Andreo a écrit :
Hi Ken,

There isn't any build-in function for that in grass, afaik. Depending on how long is your series, you could shift the map list you use in each run and then use r.univar on each output from r.series to get the minimum and by comparison, get the second minimum of the series of maps (a scalar). 

However, if what you need is a map of second minimums per pixel, I believe that might require some programming for a new function... Do you know any other software which has this function? Maybe, if there's such thing in Python for example, it could be recycled or used with grass maps in a script... Dunno, just thinking out loud

best,
Vero

El jue., 7 feb. 2019 07:16, Ken Mankoff <[hidden email]> escribió:
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

Micha Silver-2

Hello Frank:


On 2/7/19 3:16 PM, Frank David wrote:

I forgot one thing to well understand my wish. Each viewshed raster is a r.series (250-sum) of 250 r.viewshed with 2m target and 0 upto 250m terrain height. So each raster gives the hidden height of a building (250m max) anywhere on the region from a specific point of view. The purpose is to get the area of hidden wind turbine from roads for a specific wind turbine height.

Cheers,

Frank


A friend and I were working on an idea some months ago that might be of interest. Not the same as your project but the method I used is similar.


The search and rescue teams here are interested to see what areas were actually visible after they finish searching along a certain route, and what areas were not seen at all. I used the same approach as you, creating a series of overlapping r.viewshed intermediate rasters.


After seeing your post, I have added new input parameters for minimum and maximum values of overlap. The user can enter percents of minimum or maximum viewshed overlaps. Suppose I put in 20 as the minimum, then only those pixels with at least 20% of the total number of viewsheds will be shown. Similarly, if I enter 80 as the maximum, then only those pixels with less than 80% of the maximum number of viewsheds will appear in the final raster.


If you are on Gitlab, you can clone the files: No documentation yet :-(

https://gitlab.com/tsvibar/route_viewshed


If you do adopt any of this script, I'd appreciate to hear of any bugs or suggestions for improvement. If the script raises any interest, I'll add it to the Addons.


Regards,

Micha


Le 07/02/2019 à 12:48, Frank David a écrit :

Hi,

Thanks for your reply. I can have numerous raster (several hundreds). I want to find a minimum but with a tolerance, it's why I want to be able to shift the value. In fact I want to sort the value of my all input raster for each x,y cell and get the second or third, or any position in the sorted list, to build my output raster. My application is to get the area not visible from a road (I did 1600 viewshed raster from the roads every 250m). I want a tolerance in order that one point of view (top of hill) does not reduce too much the hidden area. To do so, a fixed number of my point of view (20%) does not reduce the hidden area. The result is the area is hidden from 80% of the road.

I already wrote a python script that read all rasters for each x,y cell and make a array(), sort the array values, and build a raster. It works on reasonable number of cell, but seems to fails with large number of cells. I'm not a programmer !

I wondered if this kind of treatment exists already in Grass.

If somebody wants to hemps me to develop this, I could be nice. But I think it could be an improvement of r.series function.

Sorry for my bad english, I hope you have understand my wishes !

Cheers

Frank

Le 07/02/2019 à 12:04, Veronica Andreo a écrit :
Hi Ken,

There isn't any build-in function for that in grass, afaik. Depending on how long is your series, you could shift the map list you use in each run and then use r.univar on each output from r.series to get the minimum and by comparison, get the second minimum of the series of maps (a scalar). 

However, if what you need is a map of second minimums per pixel, I believe that might require some programming for a new function... Do you know any other software which has this function? Maybe, if there's such thing in Python for example, it could be recycled or used with grass maps in a script... Dunno, just thinking out loud

best,
Vero

El jue., 7 feb. 2019 07:16, Ken Mankoff <[hidden email]> escribió:
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
cell: +972-523-665918

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

FrankD

Hi Micha,


Thank you for your email. Your script is very interesting, but I think I cannot use it because:

1- you make a sum of your viewshed list, and each viewshed value is 0 or 1 (hidden or visible). But my problem is with viewshed values from 0 upto 250 (for each viewshed this is the hidden height from a viewpoint). So make a sum has no sens.

2- you search the pourcent of visible terrain from all viewpoints, and I want find pourcent of viewpoints able to see terrain (actually to not see the maximum hidden height above terrain)

Tell me if I'm wrong...


I wrote a script with garray.array() which read the value for each specific x,y cell coordinates from all viewshed rasters, create a list of data, sort that list, and get the shifted min (according to my tolerance, for example 80% of 100 viewpoints get the twentieth smallest value). In that way Each cell of my output raster give the hidden height from 80% of my viewpoints. The script is working, but I believe that adding this function in r.series could be useful.


Cheers,

Frank


Le 15/02/2019 à 15:52, Micha Silver a écrit :

Hello Frank:


On 2/7/19 3:16 PM, Frank David wrote:

I forgot one thing to well understand my wish. Each viewshed raster is a r.series (250-sum) of 250 r.viewshed with 2m target and 0 upto 250m terrain height. So each raster gives the hidden height of a building (250m max) anywhere on the region from a specific point of view. The purpose is to get the area of hidden wind turbine from roads for a specific wind turbine height.

Cheers,

Frank


A friend and I were working on an idea some months ago that might be of interest. Not the same as your project but the method I used is similar.


The search and rescue teams here are interested to see what areas were actually visible after they finish searching along a certain route, and what areas were not seen at all. I used the same approach as you, creating a series of overlapping r.viewshed intermediate rasters.


After seeing your post, I have added new input parameters for minimum and maximum values of overlap. The user can enter percents of minimum or maximum viewshed overlaps. Suppose I put in 20 as the minimum, then only those pixels with at least 20% of the total number of viewsheds will be shown. Similarly, if I enter 80 as the maximum, then only those pixels with less than 80% of the maximum number of viewsheds will appear in the final raster.


If you are on Gitlab, you can clone the files: No documentation yet :-(

https://gitlab.com/tsvibar/route_viewshed


If you do adopt any of this script, I'd appreciate to hear of any bugs or suggestions for improvement. If the script raises any interest, I'll add it to the Addons.


Regards,

Micha


Le 07/02/2019 à 12:48, Frank David a écrit :

Hi,

Thanks for your reply. I can have numerous raster (several hundreds). I want to find a minimum but with a tolerance, it's why I want to be able to shift the value. In fact I want to sort the value of my all input raster for each x,y cell and get the second or third, or any position in the sorted list, to build my output raster. My application is to get the area not visible from a road (I did 1600 viewshed raster from the roads every 250m). I want a tolerance in order that one point of view (top of hill) does not reduce too much the hidden area. To do so, a fixed number of my point of view (20%) does not reduce the hidden area. The result is the area is hidden from 80% of the road.

I already wrote a python script that read all rasters for each x,y cell and make a array(), sort the array values, and build a raster. It works on reasonable number of cell, but seems to fails with large number of cells. I'm not a programmer !

I wondered if this kind of treatment exists already in Grass.

If somebody wants to hemps me to develop this, I could be nice. But I think it could be an improvement of r.series function.

Sorry for my bad english, I hope you have understand my wishes !

Cheers

Frank

Le 07/02/2019 à 12:04, Veronica Andreo a écrit :
Hi Ken,

There isn't any build-in function for that in grass, afaik. Depending on how long is your series, you could shift the map list you use in each run and then use r.univar on each output from r.series to get the minimum and by comparison, get the second minimum of the series of maps (a scalar). 

However, if what you need is a map of second minimums per pixel, I believe that might require some programming for a new function... Do you know any other software which has this function? Maybe, if there's such thing in Python for example, it could be recycled or used with grass maps in a script... Dunno, just thinking out loud

best,
Vero

El jue., 7 feb. 2019 07:16, Ken Mankoff <[hidden email]> escribió:
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
cell: +972-523-665918

--
Geophom
327 rue de Vieille Cour 44521 OUDON
Tel +33(0)2 85 52 02 59 - Port +33(0)6 04 47 91 06
www.geophom.fr

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: get the shifted minimum from a raster series

Micha Silver-2

On 2/16/19 12:00 PM, Frank David wrote:

Hi Micha,


Thank you for your email. Your script is very interesting, but I think I cannot use it because:

1- you make a sum of your viewshed list, and each viewshed value is 0 or 1 (hidden or visible). But my problem is with viewshed values from 0 upto 250 (for each viewshed this is the hidden height from a viewpoint). So make a sum has no sens.

2- you search the pourcent of visible terrain from all viewpoints, and I want find pourcent of viewpoints able to see terrain (actually to not see the maximum hidden height above terrain)

Tell me if I'm wrong...


You're right. That is my understanding also.

What I was thinking was to use the "max_overlap" parameter to find all areas in the search radius that are visible by at most i.e. 80% of the viewsheds. Then invert that raster - wherever there is some value, set to NULL, and wherever the value is NULL set to the DTM elevation (+ turbine height). That will give you the height at all pixels that are not visible from 20% of the viewsheds.


In any case it's an interesting application.


I wrote a script with garray.array() which read the value for each specific x,y cell coordinates from all viewshed rasters, create a list of data, sort that list, and get the shifted min (according to my tolerance, for example 80% of 100 viewpoints get the twentieth smallest value). In that way Each cell of my output raster give the hidden height from 80% of my viewpoints. The script is working, but I believe that adding this function in r.series could be useful.


Cheers,

Frank


Le 15/02/2019 à 15:52, Micha Silver a écrit :

Hello Frank:


On 2/7/19 3:16 PM, Frank David wrote:

I forgot one thing to well understand my wish. Each viewshed raster is a r.series (250-sum) of 250 r.viewshed with 2m target and 0 upto 250m terrain height. So each raster gives the hidden height of a building (250m max) anywhere on the region from a specific point of view. The purpose is to get the area of hidden wind turbine from roads for a specific wind turbine height.

Cheers,

Frank


A friend and I were working on an idea some months ago that might be of interest. Not the same as your project but the method I used is similar.


The search and rescue teams here are interested to see what areas were actually visible after they finish searching along a certain route, and what areas were not seen at all. I used the same approach as you, creating a series of overlapping r.viewshed intermediate rasters.


After seeing your post, I have added new input parameters for minimum and maximum values of overlap. The user can enter percents of minimum or maximum viewshed overlaps. Suppose I put in 20 as the minimum, then only those pixels with at least 20% of the total number of viewsheds will be shown. Similarly, if I enter 80 as the maximum, then only those pixels with less than 80% of the maximum number of viewsheds will appear in the final raster.


If you are on Gitlab, you can clone the files: No documentation yet :-(

https://gitlab.com/tsvibar/route_viewshed


If you do adopt any of this script, I'd appreciate to hear of any bugs or suggestions for improvement. If the script raises any interest, I'll add it to the Addons.


Regards,

Micha


Le 07/02/2019 à 12:48, Frank David a écrit :

Hi,

Thanks for your reply. I can have numerous raster (several hundreds). I want to find a minimum but with a tolerance, it's why I want to be able to shift the value. In fact I want to sort the value of my all input raster for each x,y cell and get the second or third, or any position in the sorted list, to build my output raster. My application is to get the area not visible from a road (I did 1600 viewshed raster from the roads every 250m). I want a tolerance in order that one point of view (top of hill) does not reduce too much the hidden area. To do so, a fixed number of my point of view (20%) does not reduce the hidden area. The result is the area is hidden from 80% of the road.

I already wrote a python script that read all rasters for each x,y cell and make a array(), sort the array values, and build a raster. It works on reasonable number of cell, but seems to fails with large number of cells. I'm not a programmer !

I wondered if this kind of treatment exists already in Grass.

If somebody wants to hemps me to develop this, I could be nice. But I think it could be an improvement of r.series function.

Sorry for my bad english, I hope you have understand my wishes !

Cheers

Frank

Le 07/02/2019 à 12:04, Veronica Andreo a écrit :
Hi Ken,

There isn't any build-in function for that in grass, afaik. Depending on how long is your series, you could shift the map list you use in each run and then use r.univar on each output from r.series to get the minimum and by comparison, get the second minimum of the series of maps (a scalar). 

However, if what you need is a map of second minimums per pixel, I believe that might require some programming for a new function... Do you know any other software which has this function? Maybe, if there's such thing in Python for example, it could be recycled or used with grass maps in a script... Dunno, just thinking out loud

best,
Vero

El jue., 7 feb. 2019 07:16, Ken Mankoff <[hidden email]> escribió:
Hi Frank,

On 2019-02-07 at 08:27 +0100, Frank David <[hidden email]>
wrote...
> I try to find how to get the shifted minimum value of a series of
> raster map. I want to shift by one, or more, the "minimum" value found
> on each cell to get the "almost minimum" of the series. Is there an
> available function/method to do that ?

I don't think I fully understand what you want. Are you working with strds? You mention "series". If I wanted to get the second-minimum of a single raster, I would use r.stats with the "-1" flag, sort, and search for the 2nd (or 2nd-to-last).

  -k.

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
cell: +972-523-665918

--
Geophom
327 rue de Vieille Cour 44521 OUDON
Tel +33(0)2 85 52 02 59 - Port +33(0)6 04 47 91 06
www.geophom.fr
-- 
Micha Silver
Ben Gurion Univ.
Sde Boker, Remote Sensing Lab
cell: +972-523-665918

_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user