[gdal-dev] NetCDF driver detected file type

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] NetCDF driver detected file type

Kurt Schwehr-2
For a custom build of gdal, I'm seeing:

  Warning 1: NetCDF driver detected file type=5, but libnetcdf detected type=3

Any idea what is causing this?  With gdal 2.3.2, from debian testing, I don't see the message.

This warning message was added here:


using:

gs://gcp-public-data-goes-17/ABI-L2-FDCF/2019/201/13/OR_ABI-L2-FDCF-M6_G17_s20192011310341_e20192011319407_c20192011319507.nc

netcdf 4.6.1
GDAL from head ~ 2018-Oct

gdalinfo NETCDF:OR_ABI-L2-FDCF-M6_G17_s20192011310341_e20192011319407_c20192011319507.nc:Area
Warning 1: NetCDF driver detected file type=5, but libnetcdf detected type=3
Driver: netCDF/Network Common Data Format
Files: none associated
Size is 5424, 5424
Coordinate System is:
PROJCS["unnamed",
    GEOGCS["unknown",
        DATUM["unknown",
            SPHEROID["Spheroid",6378137,298.2572221]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433]],
    PROJECTION["Geostationary_Satellite"],
    PARAMETER["central_meridian",-137],
    PARAMETER["satellite_height",35786023],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    EXTENSION["PROJ4","+proj=geos +lon_0=-137 +h=35786023 +x_0=0 +y_0=0 +ellps=GRS80 +units=m +no_defs  +sweep=x"]]
Origin = (-5434894.700982173904777,-5434895.218222249299288)
Pixel Size = (2004.017315487541055,2004.017315487541055)
Metadata:
  Area#add_offset=4000
  Area#cell_methods=sunglint_angle: point (no pixel produced) local_zenith_angle: point (good quality pixel produced) solar_zenith_angle: point (good quality pixel produced) t: point
  Area#coordinates=sunglint_angle local_zenith_angle solar_zenith_angle t y x
  Area#grid_mapping=goes_imager_projection
  Area#long_name=ABI L2+ Fire-Hot Spot Characterization: Fire Area
  Area#scale_factor=60.98
[SNIP]

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: NetCDF driver detected file type

Even Rouault-2
Kurt,

I would say that your netCDF build lacks libhdf5 support needed to open netCDF
4 files, and/or that your GDAL build has been done against a netCDF version
that lacked it (if you added support of HDF5 in the netCDF build, after doing
an initial build of GDAL, you also need to rebuild GDAL)

It is likely that with your build the following
gdalinfo --format netcdf | grep NETCDF_HAS_NC4=YES
reports currently nothing

Rebuild netCDF with HDF5 support, and GDAL against this new netCDF build.

Even

> For a custom build of gdal, I'm seeing:
>
>   Warning 1: NetCDF driver detected file type=5, but libnetcdf detected
> type=3
>
> Any idea what is causing this?  With gdal 2.3.2, from debian testing, I
> don't see the message.
>
> This warning message was added here:
>
> https://github.com/OSGeo/gdal/commit/813aa221af3ac13a3747f4d4a3f312f098869ad
> c
>
> using:
>
> gs://gcp-public-data-goes-17/ABI-L2-FDCF/2019/201/13/OR_ABI-L2-FDCF-M6_G17_s
> 20192011310341_e20192011319407_c20192011319507.nc
>
> netcdf 4.6.1
> HDF5 claims to be 1.11.2 and is a bit newer than
> https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse?at=807187e2f
> 200e63adb4918dd475d47546c8588e9 GDAL from head ~ 2018-Oct
>
> gdalinfo
> NETCDF:OR_ABI-L2-FDCF-M6_G17_s20192011310341_e20192011319407_c20192011319507
> .nc:Area Warning 1: NetCDF driver detected file type=5, but libnetcdf
> detected type=3 Driver: netCDF/Network Common Data Format
> Files: none associated
> Size is 5424, 5424
> Coordinate System is:
> PROJCS["unnamed",
>     GEOGCS["unknown",
>         DATUM["unknown",
>             SPHEROID["Spheroid",6378137,298.2572221]],
>         PRIMEM["Greenwich",0],
>         UNIT["degree",0.0174532925199433]],
>     PROJECTION["Geostationary_Satellite"],
>     PARAMETER["central_meridian",-137],
>     PARAMETER["satellite_height",35786023],
>     PARAMETER["false_easting",0],
>     PARAMETER["false_northing",0],
>     EXTENSION["PROJ4","+proj=geos +lon_0=-137 +h=35786023 +x_0=0 +y_0=0
> +ellps=GRS80 +units=m +no_defs  +sweep=x"]]
> Origin = (-5434894.700982173904777,-5434895.218222249299288)
> Pixel Size = (2004.017315487541055,2004.017315487541055)
> Metadata:
>   Area#add_offset=4000
>   Area#cell_methods=sunglint_angle: point (no pixel produced)
> local_zenith_angle: point (good quality pixel produced) solar_zenith_angle:
> point (good quality pixel produced) t: point
>   Area#coordinates=sunglint_angle local_zenith_angle solar_zenith_angle t y
> x
>   Area#grid_mapping=goes_imager_projection
>   Area#long_name=ABI L2+ Fire-Hot Spot Characterization: Fire Area
>   Area#scale_factor=60.98
> [SNIP]


--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: NetCDF driver detected file type

Ivan Lucena-5
Even, Kurt,

I've been thru the same issue last week. I might be wrong but the only way that I found to add support for NC4 into the netcdf-c-4.6.3 build was by enabling HDF4 .

You might try run "./configure --help | grep 4" on the netCDF SDK to see if there is another way.

I rebuild everything, HDF5, HDF4 and netCDF SDKs and GDAL and the netCDF driver and now I am not getting warnings like that anymore. 

Regards,

Ivan


From: gdal-dev <[hidden email]> on behalf of Even Rouault <[hidden email]>
Sent: Wednesday, August 7, 2019 4:39 PM
To: [hidden email] <[hidden email]>
Subject: Re: [gdal-dev] NetCDF driver detected file type
 
Kurt,

I would say that your netCDF build lacks libhdf5 support needed to open netCDF
4 files, and/or that your GDAL build has been done against a netCDF version
that lacked it (if you added support of HDF5 in the netCDF build, after doing
an initial build of GDAL, you also need to rebuild GDAL)

It is likely that with your build the following
gdalinfo --format netcdf | grep NETCDF_HAS_NC4=YES
reports currently nothing

Rebuild netCDF with HDF5 support, and GDAL against this new netCDF build.

Even

> For a custom build of gdal, I'm seeing:
>
>   Warning 1: NetCDF driver detected file type=5, but libnetcdf detected
> type=3
>
> Any idea what is causing this?  With gdal 2.3.2, from debian testing, I
> don't see the message.
>
> This warning message was added here:
>
> https://github.com/OSGeo/gdal/commit/813aa221af3ac13a3747f4d4a3f312f098869ad
> c
>
> using:
>
> gs://gcp-public-data-goes-17/ABI-L2-FDCF/2019/201/13/OR_ABI-L2-FDCF-M6_G17_s
> 20192011310341_e20192011319407_c20192011319507.nc
>
> netcdf 4.6.1
> HDF5 claims to be 1.11.2 and is a bit newer than
> https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse?at=807187e2f
> 200e63adb4918dd475d47546c8588e9 GDAL from head ~ 2018-Oct
>
> gdalinfo
> NETCDF:OR_ABI-L2-FDCF-M6_G17_s20192011310341_e20192011319407_c20192011319507
> .nc:Area Warning 1: NetCDF driver detected file type=5, but libnetcdf
> detected type=3 Driver: netCDF/Network Common Data Format
> Files: none associated
> Size is 5424, 5424
> Coordinate System is:
> PROJCS["unnamed",
>     GEOGCS["unknown",
>         DATUM["unknown",
>             SPHEROID["Spheroid",6378137,298.2572221]],
>         PRIMEM["Greenwich",0],
>         UNIT["degree",0.0174532925199433]],
>     PROJECTION["Geostationary_Satellite"],
>     PARAMETER["central_meridian",-137],
>     PARAMETER["satellite_height",35786023],
>     PARAMETER["false_easting",0],
>     PARAMETER["false_northing",0],
>     EXTENSION["PROJ4","+proj=geos +lon_0=-137 +h=35786023 +x_0=0 +y_0=0
> +ellps=GRS80 +units=m +no_defs  +sweep=x"]]
> Origin = (-5434894.700982173904777,-5434895.218222249299288)
> Pixel Size = (2004.017315487541055,2004.017315487541055)
> Metadata:
>   Area#add_offset=4000
>   Area#cell_methods=sunglint_angle: point (no pixel produced)
> local_zenith_angle: point (good quality pixel produced) solar_zenith_angle:
> point (good quality pixel produced) t: point
>   Area#coordinates=sunglint_angle local_zenith_angle solar_zenith_angle t y
> x
>   Area#grid_mapping=goes_imager_projection
>   Area#long_name=ABI L2+ Fire-Hot Spot Characterization: Fire Area
>   Area#scale_factor=60.98
> [SNIP]


--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: NetCDF driver detected file type

Kurt Schwehr-2
Hmm... these are netcdf 5 according to file.  Does that not imply that that I don't need to worry about NC4? 

file *.nc
OR_ABI-L2-FDCF-M6_G17_s20192011300341_e20192011309408_c20192011309501.nc: Hierarchical Data Format (version 5) data
OR_ABI-L2-FDCF-M6_G17_s20192011310341_e20192011319407_c20192011319507.nc: Hierarchical Data Format (version 5) data
OR_ABI-L2-FDCF-M6_G17_s20192011320341_e20192011329408_c20192011329487.nc: Hierarchical Data Format (version 5) data
OR_ABI-L2-FDCF-M6_G17_s20192011330341_e20192011339408_c20192011339497.nc: Hierarchical Data Format (version 5) data
OR_ABI-L2-FDCF-M6_G17_s20192011340341_e20192011349408_c20192011349492.nc: Hierarchical Data Format (version 5) data
OR_ABI-L2-FDCF-M6_G17_s20192011350341_e20192011359408_c20192011359486.nc: Hierarchical Data Format (version 5) data

I have the extra fun that my build is bazel based.  I do have:

    "-DFRMT_hdf4",
    "-DFRMT_hdf5",
    "-DFRMT_netcdf",

This indeed give nothing:

gdalinfo --format netcdf | grep NETCDF_HAS_NC4

I have more digging to do on the config checking side of things.  I don't have the dispatch tables for hdf4 in netcdf.

On Wed, Aug 7, 2019 at 1:51 PM Ivan Lucena <[hidden email]> wrote:
Even, Kurt,

I've been thru the same issue last week. I might be wrong but the only way that I found to add support for NC4 into the netcdf-c-4.6.3 build was by enabling HDF4 .

You might try run "./configure --help | grep 4" on the netCDF SDK to see if there is another way.

I rebuild everything, HDF5, HDF4 and netCDF SDKs and GDAL and the netCDF driver and now I am not getting warnings like that anymore. 

Regards,

Ivan


From: gdal-dev <[hidden email]> on behalf of Even Rouault <[hidden email]>
Sent: Wednesday, August 7, 2019 4:39 PM
To: [hidden email] <[hidden email]>
Subject: Re: [gdal-dev] NetCDF driver detected file type
 
Kurt,

I would say that your netCDF build lacks libhdf5 support needed to open netCDF
4 files, and/or that your GDAL build has been done against a netCDF version
that lacked it (if you added support of HDF5 in the netCDF build, after doing
an initial build of GDAL, you also need to rebuild GDAL)

It is likely that with your build the following
gdalinfo --format netcdf | grep NETCDF_HAS_NC4=YES
reports currently nothing

Rebuild netCDF with HDF5 support, and GDAL against this new netCDF build.

Even

> For a custom build of gdal, I'm seeing:
>
>   Warning 1: NetCDF driver detected file type=5, but libnetcdf detected
> type=3
>
> Any idea what is causing this?  With gdal 2.3.2, from debian testing, I
> don't see the message.
>
> This warning message was added here:
>
> https://github.com/OSGeo/gdal/commit/813aa221af3ac13a3747f4d4a3f312f098869ad
> c
>
> using:
>
> gs://gcp-public-data-goes-17/ABI-L2-FDCF/2019/201/13/OR_ABI-L2-FDCF-M6_G17_s
> 20192011310341_e20192011319407_c20192011319507.nc
>
> netcdf 4.6.1
> HDF5 claims to be 1.11.2 and is a bit newer than
> https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse?at=807187e2f
> 200e63adb4918dd475d47546c8588e9 GDAL from head ~ 2018-Oct
>
> gdalinfo
> NETCDF:OR_ABI-L2-FDCF-M6_G17_s20192011310341_e20192011319407_c20192011319507
> .nc:Area Warning 1: NetCDF driver detected file type=5, but libnetcdf
> detected type=3 Driver: netCDF/Network Common Data Format
> Files: none associated
> Size is 5424, 5424
> Coordinate System is:
> PROJCS["unnamed",
>     GEOGCS["unknown",
>         DATUM["unknown",
>             SPHEROID["Spheroid",6378137,298.2572221]],
>         PRIMEM["Greenwich",0],
>         UNIT["degree",0.0174532925199433]],
>     PROJECTION["Geostationary_Satellite"],
>     PARAMETER["central_meridian",-137],
>     PARAMETER["satellite_height",35786023],
>     PARAMETER["false_easting",0],
>     PARAMETER["false_northing",0],
>     EXTENSION["PROJ4","+proj=geos +lon_0=-137 +h=35786023 +x_0=0 +y_0=0
> +ellps=GRS80 +units=m +no_defs  +sweep=x"]]
> Origin = (-5434894.700982173904777,-5434895.218222249299288)
> Pixel Size = (2004.017315487541055,2004.017315487541055)
> Metadata:
>   Area#add_offset=4000
>   Area#cell_methods=sunglint_angle: point (no pixel produced)
> local_zenith_angle: point (good quality pixel produced) solar_zenith_angle:
> point (good quality pixel produced) t: point
>   Area#coordinates=sunglint_angle local_zenith_angle solar_zenith_angle t y
> x
>   Area#grid_mapping=goes_imager_projection
>   Area#long_name=ABI L2+ Fire-Hot Spot Characterization: Fire Area
>   Area#scale_factor=60.98
> [SNIP]


--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: NetCDF driver detected file type

Even Rouault-2
On mercredi 7 août 2019 14:24:32 CEST Kurt Schwehr wrote:
> Hmm... these are netcdf 5 according to file.

HDF5, not netCDF 5. netCDF v4 file format is a profile on top of HDF5.

> I have the extra fun that my build is bazel based.

Ah that likely explains it... You need to add -DNETCDF_HAS_NC4 to the
compilation flags of the frmts/netcdf/*.cpp files


--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: NetCDF driver detected file type

Kurt Schwehr-2
Thanks!  Adding -DNETCDF_HAS_NC4 makes the warning go away.  

Now I wonder what that means to users of this code that have been living without that for forever.

I will try gdal_translate with and without that flag and see what happens.  It looks like I should see differences with unsigned types.

On Wed, Aug 7, 2019 at 2:36 PM Even Rouault <[hidden email]> wrote:
On mercredi 7 août 2019 14:24:32 CEST Kurt Schwehr wrote:
> Hmm... these are netcdf 5 according to file.

HDF5, not netCDF 5. netCDF v4 file format is a profile on top of HDF5.

> I have the extra fun that my build is bazel based.

Ah that likely explains it... You need to add -DNETCDF_HAS_NC4 to the
compilation flags of the frmts/netcdf/*.cpp files


--
Spatialys - Geospatial professional services
http://www.spatialys.com

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev