Re: grass-user Digest, Vol 105, Issue 36

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: grass-user Digest, Vol 105, Issue 36

Maci Edwards
Please unsubscribe me. Thank you!

On Tue, Jan 20, 2015 at 2:00 PM, <[hidden email]> wrote:
Send grass-user mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.osgeo.org/mailman/listinfo/grass-user
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of grass-user digest..."


Today's Topics:

   1. GRASS/gdal grib2 decoding issue for new NCEP MRMS data
      (Thomas Adams)
   2. Re: Large Grass Raster Notes (Markus Neteler)
   3. Re: Large Grass Raster Notes (Vaclav Petras)


----------------------------------------------------------------------

Message: 1
Date: Mon, 19 Jan 2015 21:57:56 -0700
From: Thomas Adams <[hidden email]>
To: "[hidden email]" <[hidden email]>
Subject: [GRASS-user] GRASS/gdal grib2 decoding issue for new NCEP
        MRMS data
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

All:

I'm not on the [hidden email] mailing list, so I can not report
this here. NOAA/NCEP is producing new hourly CONUS NEXRAD radar-based
meteorological data -- my interest is precipitation. The data is available
now via a UCAR LDM process, described in the 1st link. The problem is
(using the downloadable sample data), gdal recognizes the data files as
grib2, but it will not decode the data (I have not previously had any
problems). The Google docs file states that there is an issue and points to
an updated wgrib2 that can handle the grib2 decoding (which it does) -- the
second link.

wgrib2 v0.2.0.1 7/2014 Wesley Ebisuzaki, Reinoud Bokhorst, Jaakko Hyv?tti,
Dusan Jovic, Kristian Nilssen, Karl Pfeiffer, Pablo Romero, Manfred
Schwarb, Arlindo da Silva, Niklas Sondell, Sergey Varlamov

https://docs.google.com/document/d/1LeVcn_taIXZgzZb5JgWqaVr0xVs7GmA6RpHcb8ZGiwk/edit

http://www.cpc.noaa.gov/products/wesley/wgrib2/

When I used wgrib2 to produce a binary file, I could then import the
example data file into GRASS:

r.info map=MRMS_GaugeCorr_QPE_01H_00.00_20141208_120000@teaiii

 +----------------------------------------------------------------------------+
 | Map:      MRMS_GaugeCorr_QPE_01H_00.00_  Date: Mon Jan 19 20:36:05
2015    |
 | Mapset:   teaiii                         Login of Creator:
teaiii          |
 | Location:
world_location                                                   |
 | DataBase:
/home/teaiii/grass/data                                          |
 | Title:     ( MRMS_GaugeCorr_QPE_01H_00.00_20141208-120000
)                |
 | Timestamp:
none                                                            |
 |----------------------------------------------------------------------------|
 |
|
 |   Type of Map:  raster               Number of Categories:
0               |
 |   Data Type:
FCELL                                                      |
 |   Rows:
3500                                                       |
 |   Columns:
7000                                                       |
 |   Total Cells:
24500000                                                   |
 |        Projection:
Latitude-Longitude                                      |
 |            N:        55N    S:        20N   Res:
0:00:36                   |
 |            E:        60W    W:       130W   Res:
0:00:36                   |
 |   Range of data:    min = 0  max =
210.5                                   |
 |
|
 |   Data
Description:                                                        |
 |    generated by
r.in.bin                                                   |
 |
|
 |
Comments:                                                                |
 |    r.in.bin -f
input="/home/teaiii/Downloads/MRMS_GRIB2_SAMPLE-precip/t\   |
 |    est.bin" output="MRMS_GaugeCorr_QPE_01H_00.00_20141208-120000"
bytes\   |
 |    =4 header=8 bands=1 order="native" north=55 south=20 east=-60
west=-\   |
 |    130 rows=3500 cols=7000 anull=-3
flip="v"                               |
 |
|
 +----------------------------------------------------------------------------+

So, I thought the community would be interested in (1) knowing about the
data and (2) the GRASS/gdal decoding issue.

Cheers!
Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150119/2ce8984c/attachment-0001.html>

------------------------------

Message: 2
Date: Tue, 20 Jan 2015 17:35:48 +0100
From: Markus Neteler <[hidden email]>
To: Jeshua Lacock <[hidden email]>
Cc: GRASS user list <[hidden email]>
Subject: Re: [GRASS-user] Large Grass Raster Notes
Message-ID:
        <CALFmHhuKm1=[hidden email]>
Content-Type: text/plain; charset=UTF-8

On Sun, Jan 18, 2015 at 1:00 AM, Jeshua Lacock <[hidden email]> wrote:
>
> Greetings,
>
> In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the  GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode.
>
> 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki.
>
> 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery.

You may check/benchmark the other resampling modules, some of them may
be more modern (ideally faster):

http://grass.osgeo.org/grass70/manuals/keywords.html#resample

In case you try, please share the outcome.

cheers,
Markus


------------------------------

Message: 3
Date: Tue, 20 Jan 2015 14:04:05 -0500
From: Vaclav Petras <[hidden email]>
To: Jeshua Lacock <[hidden email]>
Cc: GRASS user list <[hidden email]>
Subject: Re: [GRASS-user] Large Grass Raster Notes
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

On Sun, Jan 18, 2015 at 1:03 PM, Jeshua Lacock <[hidden email]> wrote:

> Actually, I misspoke, the wildcard will not work with long lists either as
> it will crash with some kind argument too long message as well.
>

This is strange, the wild card is interpreted inside the module, so the
argument list shouldn't be too long. If you don't quite the pattern the
shell can do the expansion but this is not what you want.

Correct:

g.remove ... pattern="landsat_*"

Incorrect:

g.remove ... pattern=landsat_*

The later will transform the command line to:

g.remove ... pattern=landsat_01 landsat_02 landsat_03 landsat_04 ...

in case you are in directory with files or directories named landsat_01,
landsat_02, landsat_03, ...

What is the message you get? And what is the command return code (do echo
$? to find out)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150120/6678f2d5/attachment-0001.html>

------------------------------

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

End of grass-user Digest, Vol 105, Issue 36
*******************************************



--
Desire can be of greater import than utility; experience more important than closure.

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user
Reply | Threaded
Open this post in threaded view
|

Re: grass-user Digest, Vol 105, Issue 36

Anna Petrášová
You can do it easily yourself. Please follow the instructions on the bottom of the page:

Thank you!

Anna

On Tue, Jan 20, 2015 at 4:54 PM, Maci Edwards <[hidden email]> wrote:
Please unsubscribe me. Thank you!

On Tue, Jan 20, 2015 at 2:00 PM, <[hidden email]> wrote:
Send grass-user mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.osgeo.org/mailman/listinfo/grass-user
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of grass-user digest..."


Today's Topics:

   1. GRASS/gdal grib2 decoding issue for new NCEP MRMS data
      (Thomas Adams)
   2. Re: Large Grass Raster Notes (Markus Neteler)
   3. Re: Large Grass Raster Notes (Vaclav Petras)


----------------------------------------------------------------------

Message: 1
Date: Mon, 19 Jan 2015 21:57:56 -0700
From: Thomas Adams <[hidden email]>
To: "[hidden email]" <[hidden email]>
Subject: [GRASS-user] GRASS/gdal grib2 decoding issue for new NCEP
        MRMS data
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

All:

I'm not on the [hidden email] mailing list, so I can not report
this here. NOAA/NCEP is producing new hourly CONUS NEXRAD radar-based
meteorological data -- my interest is precipitation. The data is available
now via a UCAR LDM process, described in the 1st link. The problem is
(using the downloadable sample data), gdal recognizes the data files as
grib2, but it will not decode the data (I have not previously had any
problems). The Google docs file states that there is an issue and points to
an updated wgrib2 that can handle the grib2 decoding (which it does) -- the
second link.

wgrib2 v0.2.0.1 7/2014 Wesley Ebisuzaki, Reinoud Bokhorst, Jaakko Hyv?tti,
Dusan Jovic, Kristian Nilssen, Karl Pfeiffer, Pablo Romero, Manfred
Schwarb, Arlindo da Silva, Niklas Sondell, Sergey Varlamov

https://docs.google.com/document/d/1LeVcn_taIXZgzZb5JgWqaVr0xVs7GmA6RpHcb8ZGiwk/edit

http://www.cpc.noaa.gov/products/wesley/wgrib2/

When I used wgrib2 to produce a binary file, I could then import the
example data file into GRASS:

r.info map=MRMS_GaugeCorr_QPE_01H_00.00_20141208_120000@teaiii

 +----------------------------------------------------------------------------+
 | Map:      MRMS_GaugeCorr_QPE_01H_00.00_  Date: Mon Jan 19 20:36:05
2015    |
 | Mapset:   teaiii                         Login of Creator:
teaiii          |
 | Location:
world_location                                                   |
 | DataBase:
/home/teaiii/grass/data                                          |
 | Title:     ( MRMS_GaugeCorr_QPE_01H_00.00_20141208-120000
)                |
 | Timestamp:
none                                                            |
 |----------------------------------------------------------------------------|
 |
|
 |   Type of Map:  raster               Number of Categories:
0               |
 |   Data Type:
FCELL                                                      |
 |   Rows:
3500                                                       |
 |   Columns:
7000                                                       |
 |   Total Cells:
24500000                                                   |
 |        Projection:
Latitude-Longitude                                      |
 |            N:        55N    S:        20N   Res:
0:00:36                   |
 |            E:        60W    W:       130W   Res:
0:00:36                   |
 |   Range of data:    min = 0  max =
210.5                                   |
 |
|
 |   Data
Description:                                                        |
 |    generated by
r.in.bin                                                   |
 |
|
 |
Comments:                                                                |
 |    r.in.bin -f
input="/home/teaiii/Downloads/MRMS_GRIB2_SAMPLE-precip/t\   |
 |    est.bin" output="MRMS_GaugeCorr_QPE_01H_00.00_20141208-120000"
bytes\   |
 |    =4 header=8 bands=1 order="native" north=55 south=20 east=-60
west=-\   |
 |    130 rows=3500 cols=7000 anull=-3
flip="v"                               |
 |
|
 +----------------------------------------------------------------------------+

So, I thought the community would be interested in (1) knowing about the
data and (2) the GRASS/gdal decoding issue.

Cheers!
Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150119/2ce8984c/attachment-0001.html>

------------------------------

Message: 2
Date: Tue, 20 Jan 2015 17:35:48 +0100
From: Markus Neteler <[hidden email]>
To: Jeshua Lacock <[hidden email]>
Cc: GRASS user list <[hidden email]>
Subject: Re: [GRASS-user] Large Grass Raster Notes
Message-ID:
        <CALFmHhuKm1=[hidden email]>
Content-Type: text/plain; charset=UTF-8

On Sun, Jan 18, 2015 at 1:00 AM, Jeshua Lacock <[hidden email]> wrote:
>
> Greetings,
>
> In the event my experience working with rather large GRASS rasters may be useful, I thought I would share it. The following notes were compiled running the  GRASS 7.0.0beta4-141230 package on Mac OS X 10.10.1. The system is a 6-core Xeon with 32GB of RAM, running in full 64-bit mode.
>
> 1. Using an r.external virtual mosaic as the input proved impossible trying to run r.resample on it on trying to produce a nearly teracell raster (I ended up breaking sub-tiles). I closely followed the instructions on the Wiki.
>
> 2. On smaller sub-tiles of said image, r.resample was insanely slow even with the external imagery.

You may check/benchmark the other resampling modules, some of them may
be more modern (ideally faster):

http://grass.osgeo.org/grass70/manuals/keywords.html#resample

In case you try, please share the outcome.

cheers,
Markus


------------------------------

Message: 3
Date: Tue, 20 Jan 2015 14:04:05 -0500
From: Vaclav Petras <[hidden email]>
To: Jeshua Lacock <[hidden email]>
Cc: GRASS user list <[hidden email]>
Subject: Re: [GRASS-user] Large Grass Raster Notes
Message-ID:
        <[hidden email]>
Content-Type: text/plain; charset="utf-8"

On Sun, Jan 18, 2015 at 1:03 PM, Jeshua Lacock <[hidden email]> wrote:

> Actually, I misspoke, the wildcard will not work with long lists either as
> it will crash with some kind argument too long message as well.
>

This is strange, the wild card is interpreted inside the module, so the
argument list shouldn't be too long. If you don't quite the pattern the
shell can do the expansion but this is not what you want.

Correct:

g.remove ... pattern="landsat_*"

Incorrect:

g.remove ... pattern=landsat_*

The later will transform the command line to:

g.remove ... pattern=landsat_01 landsat_02 landsat_03 landsat_04 ...

in case you are in directory with files or directories named landsat_01,
landsat_02, landsat_03, ...

What is the message you get? And what is the command return code (do echo
$? to find out)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150120/6678f2d5/attachment-0001.html>

------------------------------

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

End of grass-user Digest, Vol 105, Issue 36
*******************************************



--
Desire can be of greater import than utility; experience more important than closure.

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user


_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user