[gdal-dev] GRIB driver updates / call for testing / TDLPACK

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] GRIB driver updates / call for testing / TDLPACK

Even Rouault-2

Hi,

 

I wanted to inform you that the GRIB driver has received significant updates lately in trunk :

* the underlying degrib and g2clib libraries used to do the GRIB1 & GRIB2 parsing have been updated to their latest upstream versions (*). Given that they are patched in GDAL for various reasons (virtual I/O capabilities, security fixes, other fixes), the update involved merging, and thus regressions might potentially have sliped in.

* various improvements on top of that:

- faster gdalinfo on GRIB2 products on multiband datasets (no need to decompress the grid data of the band other than the first one)

- avoid erroring out on unsupported product definition template

- fix in the GRIB1 reader that should make it possible to read more products (those with NV,the number of vertical coordinate parameters, different from 0, which causes previously error messages like "GRIB1 GDS: Expect PV = 255 !=33")

* addition of GRIB2 write support

 

So I'd appreciate testing from users to confirm if products they managed to read with past versions are still readable with trunk.

 

I'm wondering if anyone relies on the support for the TDLPACK that apparently the degrib library supports. This isn't even documented in the

http://gdal.org/frmt_grib.html help page. From what I can read in

https://github.com/eengl/pytdlpack , it seems that TDLPACK is mostly an internal format of NOAA/NWS Meteorological Development Lab. Couldn't find any publicly available file in that format. If nobody uses this capability, I'd prefer we comment out all the parts related to TDLPack so as to reduce the vulnerability surface.

 

Even

 

(*) https://svn.osgeo.org/gdal/trunk/gdal/frmts/grib/degrib/README.TXT

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: GRIB driver updates / call for testing / TDLPACK

Kurt Schwehr-2
I'm pretty much up to head in production and testing is doing fine.  But... It's only been a couple hours.  If there is nothing from me in the next day or two, I give it a +1.  Thanks for the hard work!

My first pass check with TDLPACK is that I know of nobody who uses it.

On Mon, Dec 4, 2017 at 11:46 AM, Even Rouault <[hidden email]> wrote:

Hi,

 

I wanted to inform you that the GRIB driver has received significant updates lately in trunk :

* the underlying degrib and g2clib libraries used to do the GRIB1 & GRIB2 parsing have been updated to their latest upstream versions (*). Given that they are patched in GDAL for various reasons (virtual I/O capabilities, security fixes, other fixes), the update involved merging, and thus regressions might potentially have sliped in.

* various improvements on top of that:

- faster gdalinfo on GRIB2 products on multiband datasets (no need to decompress the grid data of the band other than the first one)

- avoid erroring out on unsupported product definition template

- fix in the GRIB1 reader that should make it possible to read more products (those with NV,the number of vertical coordinate parameters, different from 0, which causes previously error messages like "GRIB1 GDS: Expect PV = 255 !=33")

* addition of GRIB2 write support

 

So I'd appreciate testing from users to confirm if products they managed to read with past versions are still readable with trunk.

 

I'm wondering if anyone relies on the support for the TDLPACK that apparently the degrib library supports. This isn't even documented in the

http://gdal.org/frmt_grib.html help page. From what I can read in

https://github.com/eengl/pytdlpack , it seems that TDLPACK is mostly an internal format of NOAA/NWS Meteorological Development Lab. Couldn't find any publicly available file in that format. If nobody uses this capability, I'd prefer we comment out all the parts related to TDLPack so as to reduce the vulnerability surface.

 

Even

 

(*) https://svn.osgeo.org/gdal/trunk/gdal/frmts/grib/degrib/README.TXT

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev



--

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: GRIB driver updates / call for testing / TDLPACK

Kurt Schwehr-2
I've found nothing that depends on TDLPACK

On Mon, Dec 4, 2017 at 12:32 PM, Kurt Schwehr <[hidden email]> wrote:
I'm pretty much up to head in production and testing is doing fine.  But... It's only been a couple hours.  If there is nothing from me in the next day or two, I give it a +1.  Thanks for the hard work!

My first pass check with TDLPACK is that I know of nobody who uses it.

On Mon, Dec 4, 2017 at 11:46 AM, Even Rouault <[hidden email]> wrote:

Hi,

 

I wanted to inform you that the GRIB driver has received significant updates lately in trunk :

* the underlying degrib and g2clib libraries used to do the GRIB1 & GRIB2 parsing have been updated to their latest upstream versions (*). Given that they are patched in GDAL for various reasons (virtual I/O capabilities, security fixes, other fixes), the update involved merging, and thus regressions might potentially have sliped in.

* various improvements on top of that:

- faster gdalinfo on GRIB2 products on multiband datasets (no need to decompress the grid data of the band other than the first one)

- avoid erroring out on unsupported product definition template

- fix in the GRIB1 reader that should make it possible to read more products (those with NV,the number of vertical coordinate parameters, different from 0, which causes previously error messages like "GRIB1 GDS: Expect PV = 255 !=33")

* addition of GRIB2 write support

 

So I'd appreciate testing from users to confirm if products they managed to read with past versions are still readable with trunk.

 

I'm wondering if anyone relies on the support for the TDLPACK that apparently the degrib library supports. This isn't even documented in the

http://gdal.org/frmt_grib.html help page. From what I can read in

https://github.com/eengl/pytdlpack , it seems that TDLPACK is mostly an internal format of NOAA/NWS Meteorological Development Lab. Couldn't find any publicly available file in that format. If nobody uses this capability, I'd prefer we comment out all the parts related to TDLPack so as to reduce the vulnerability surface.

 

Even

 

(*) https://svn.osgeo.org/gdal/trunk/gdal/frmts/grib/degrib/README.TXT

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev



--



--

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: GRIB driver updates / call for testing / TDLPACK

Even Rouault-2

On mardi 5 décembre 2017 07:15:39 CET Kurt Schwehr wrote:

> I've found nothing that depends on TDLPACK

 

I've just committed the change to remove TDLPACK support from compilation

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: GRIB driver updates / call for testing / TDLPACK

Kurt Schwehr-2
What about metaprint?  Can the TDLP_MERCATOR, TDLP_POLAR, and TDLP_LAMBERT go away?   And then I think that is the last use of tdlpack.h

On Fri, Dec 8, 2017 at 9:34 AM, Even Rouault <[hidden email]> wrote:

On mardi 5 décembre 2017 07:15:39 CET Kurt Schwehr wrote:

> I've found nothing that depends on TDLPACK

 

I've just committed the change to remove TDLPACK support from compilation

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com




--

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: GRIB driver updates / call for testing / TDLPACK

Even Rouault-2

On vendredi 8 décembre 2017 13:33:11 CET Kurt Schwehr wrote:

> What about metaprint? Can the TDLP_MERCATOR, TDLP_POLAR, and TDLP_LAMBERT

> go away? And then I think that is the last use of tdlpack.h

 

The code in metaprint.cpp that uses TDLP_ was already commented out

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: GRIB driver updates / call for testing / TDLPACK

Kurt Schwehr-2
Ah, then just the #include "tdlpack.h" in metaprint needs to get commentout, wrapped, or removed


On Fri, Dec 8, 2017 at 1:39 PM, Even Rouault <[hidden email]> wrote:

On vendredi 8 décembre 2017 13:33:11 CET Kurt Schwehr wrote:

> What about metaprint? Can the TDLP_MERCATOR, TDLP_POLAR, and TDLP_LAMBERT

> go away? And then I think that is the last use of tdlpack.h

 

The code in metaprint.cpp that uses TDLP_ was already commented out

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com




--

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev