Quantcast

Aggregation of massive number of raster layers with r.series

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Aggregation of massive number of raster layers with r.series

Pierre Roudier
Hi,

I am trying to compute the 95th percentile of a massive grid (12+
million pixels) for a massive number of layers (~2500 layers).

I am doing the aggregation using r.series on our cluster running grass
7.2, but of course it takes ages (21% there after 3 days).

- I tried to tile the process, but it doesn't seem to help much.

- Is there any benefit for me to switch to t.rast.aggregate? My
understanding was that it was a wrapper around r.series.

- Does anyone have a fancy trick to make the aggregation go faster
(parallelisation)?

Cheers,

Pierre
_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Aggregation of massive number of raster layers with r.series

Robert Brown
Have you tried r.mapcalc using "nested if statements"?

Sent from my iPhone

> On Apr 20, 2017, at 6:50 PM, Pierre Roudier <[hidden email]> wrote:
>
> Hi,
>
> I am trying to compute the 95th percentile of a massive grid (12+
> million pixels) for a massive number of layers (~2500 layers).
>
> I am doing the aggregation using r.series on our cluster running grass
> 7.2, but of course it takes ages (21% there after 3 days).
>
> - I tried to tile the process, but it doesn't seem to help much.
>
> - Is there any benefit for me to switch to t.rast.aggregate? My
> understanding was that it was a wrapper around r.series.
>
> - Does anyone have a fancy trick to make the aggregation go faster
> (parallelisation)?
>
> Cheers,
>
> Pierre
> _______________________________________________
> grass-user mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/grass-user
_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
SBL
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Aggregation of massive number of raster layers with r.series

SBL
In reply to this post by Pierre Roudier
Hi Pierre,

tiling should speed up significantly, if you process the tiles in parallel (and if you have multiple cores and if IO is not the bottleneck (e.g. slow network connection to the data)).
Care has to be taken with the region settings, though.

See e.g.:
https://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Working_with_tiles

Cheers
Stefan

________________________________________
Von: grass-user <[hidden email]> im Auftrag von Pierre Roudier <[hidden email]>
Gesendet: Freitag, 21. April 2017 00:49
An: grass-user
Betreff: [GRASS-user] Aggregation of massive number of raster layers with       r.series

Hi,

I am trying to compute the 95th percentile of a massive grid (12+
million pixels) for a massive number of layers (~2500 layers).

I am doing the aggregation using r.series on our cluster running grass
7.2, but of course it takes ages (21% there after 3 days).

- I tried to tile the process, but it doesn't seem to help much.

- Is there any benefit for me to switch to t.rast.aggregate? My
understanding was that it was a wrapper around r.series.

- Does anyone have a fancy trick to make the aggregation go faster
(parallelisation)?

Cheers,

Pierre
_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
_______________________________________________
grass-user mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/grass-user
Loading...