[gdal-dev] libcurl and the certificates and Windows

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] libcurl and the certificates and Windows

Joaquim Luis
Hi,

For quite some time I cannot use the 'vsis' because of certificates issue.  
For example, a GMT test that has a command like this no longer works on  
Windows

gdalinfo  
/vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbit.jpg

because

ERROR 11: HTTP response code: 301 - SSL certificate problem: unable to get  
local issuer certificate
gdalinfo failed - unable to open  
'/vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbit.jpg'.

It used to work but probably with an older libcurl dll.
The above is with my own build gdal and dependencies (libcurl included)  
but the same happens with the gisinternals binaries.

I have re(and re)ad this page about the certificates

https://curl.haxx.se/docs/sslcerts.html

but regarding Windows and the curl-ca-bundle.crt file what is said about  
it simply does not work. The only thing that works is setting the ENV  
variable

set CURL_CA_BUNDLE=V:\bin\curl-ca-bundle.crt

Now, we had this in GMT recently and I used the nuke option

        curl_easy_setopt (Curl, CURLOPT_SSL_VERIFYPEER, 0L); /* Tell libcurl to  
not verify the peer */

so tried to do the same thing in the GDAL code (the obvious point seamed  
to be VSICurlSetOptions in cpl_vsi_curl.cpp) but still does not work.

OSGeo4W works but probably because they are still using a 4 years old  
libcurl.dll

Am I the only one seeing this?

Thanks

Joaquim
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Even Rouault-2

On samedi 3 juin 2017 17:04:07 CEST Joaquim Luis wrote:

> Hi,

>

> For quite some time I cannot use the 'vsis' because of certificates issue.

> For example, a GMT test that has a command like this no longer works on

> Windows

>

> gdalinfo

> /vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbit

> .jpg

>

> because

>

> ERROR 11: HTTP response code: 301 - SSL certificate problem: unable to get

> local issuer certificate

> gdalinfo failed - unable to open

> '/vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbi

> t.jpg'.

>

> It used to work but probably with an older libcurl dll.

> The above is with my own build gdal and dependencies (libcurl included)

> but the same happens with the gisinternals binaries.

>

> I have re(and re)ad this page about the certificates

>

> https://curl.haxx.se/docs/sslcerts.html

>

> but regarding Windows and the curl-ca-bundle.crt file what is said about

> it simply does not work. The only thing that works is setting the ENV

> variable

>

> set CURL_CA_BUNDLE=V:\bin\curl-ca-bundle.crt

>

> Now, we had this in GMT recently and I used the nuke option

>

> curl_easy_setopt (Curl, CURLOPT_SSL_VERIFYPEER, 0L); /* Tell libcurl to

> not verify the peer */

>

> so tried to do the same thing in the GDAL code (the obvious point seamed

> to be VSICurlSetOptions in cpl_vsi_curl.cpp) but still does not work.

 

Someone reported to me a similar issue with recent OSGeo4W.

 

Did you try setting GDAL_HTTP_UNSAFESSL=YES? This is taken into account in CPLHTTPSetOptions() that is called by VSICurlSetOptions(), and this set CURLOPT_SSL_VERIFYPEER=0 and CURLOPT_SSL_VERIFYHOST=0.

 

This solved the issue.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Joaquim Luis
On Sat, 03 Jun 2017 17:22:33 +0100, Even Rouault <[hidden email]> wrote:

On samedi 3 juin 2017 17:04:07 CEST Joaquim Luis wrote:

> Hi,

>

> For quite some time I cannot use the 'vsis' because of certificates issue.

> For example, a GMT test that has a command like this no longer works on

> Windows

>

> gdalinfo

> /vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbit

> .jpg

>

> because

>

> ERROR 11: HTTP response code: 301 - SSL certificate problem: unable to get

> local issuer certificate

> gdalinfo failed - unable to open

> '/vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbi

> t.jpg'.

>

> It used to work but probably with an older libcurl dll.

> The above is with my own build gdal and dependencies (libcurl included)

> but the same happens with the gisinternals binaries.

>

> I have re(and re)ad this page about the certificates

>

> https://curl.haxx.se/docs/sslcerts.html

>

> but regarding Windows and the curl-ca-bundle.crt file what is said about

> it simply does not work. The only thing that works is setting the ENV

> variable

>

> set CURL_CA_BUNDLE=V:\bin\curl-ca-bundle.crt

>

> Now, we had this in GMT recently and I used the nuke option

>

> curl_easy_setopt (Curl, CURLOPT_SSL_VERIFYPEER, 0L); /* Tell libcurl to

> not verify the peer */

>

> so tried to do the same thing in the GDAL code (the obvious point seamed

> to be VSICurlSetOptions in cpl_vsi_curl.cpp) but still does not work.

 

Someone reported to me a similar issue with recent OSGeo4W.

 

Did you try setting GDAL_HTTP_UNSAFESSL=YES? This is taken into account in CPLHTTPSetOptions() that is called by VSICurlSetOptions(), and this set CURLOPT_SSL_VERIFYPEER=0 and CURLOPT_SSL_VERIFYHOST=0.

 

This solved the issue.

 


Thanks, yes that works too (and, no I hadn't tried it before) although it's a different solution than setting  CURL_CA_BUNDLE , which does not turn out the certificates verification. 

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Joaquim Luis
For reference

https://github.com/curl/curl/issues/1538


On Sat, 03 Jun 2017 17:22:33 +0100, Even Rouault <[hidden email]> wrote:

On samedi 3 juin 2017 17:04:07 CEST Joaquim Luis wrote:

> Hi,

>

> For quite some time I cannot use the 'vsis' because of certificates issue.

> For example, a GMT test that has a command like this no longer works on

> Windows

>

> gdalinfo

> /vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbit

> .jpg

>

> because

>

> ERROR 11: HTTP response code: 301 - SSL certificate problem: unable to get

> local issuer certificate

> gdalinfo failed - unable to open

> '/vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbi

> t.jpg'.

>

> It used to work but probably with an older libcurl dll.

> The above is with my own build gdal and dependencies (libcurl included)

> but the same happens with the gisinternals binaries.

>

> I have re(and re)ad this page about the certificates

>

> https://curl.haxx.se/docs/sslcerts.html

>

> but regarding Windows and the curl-ca-bundle.crt file what is said about

> it simply does not work. The only thing that works is setting the ENV

> variable

>

> set CURL_CA_BUNDLE=V:\bin\curl-ca-bundle.crt

>

> Now, we had this in GMT recently and I used the nuke option

>

> curl_easy_setopt (Curl, CURLOPT_SSL_VERIFYPEER, 0L); /* Tell libcurl to

> not verify the peer */

>

> so tried to do the same thing in the GDAL code (the obvious point seamed

> to be VSICurlSetOptions in cpl_vsi_curl.cpp) but still does not work.

 

Someone reported to me a similar issue with recent OSGeo4W.

 

Did you try setting GDAL_HTTP_UNSAFESSL=YES? This is taken into account in CPLHTTPSetOptions() that is called by VSICurlSetOptions(), and this set CURLOPT_SSL_VERIFYPEER=0 and CURLOPT_SSL_VERIFYHOST=0.

 

This solved the issue.

 


Thanks, yes that works too (and, no I hadn't tried it before) although it's a different solution than setting  CURL_CA_BUNDLE , which does not turn out the certificates verification. 




_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Vautour, André (INT)

I’d like to add that I think an option like GDAL_HTTP_CA_CERT_FILE or GDAL_HTTP_CA_CERT_PATH would be useful to have.

 

In our applications, usage of libcurl outside of GDAL sets the CURLOPT_CAINFO to point to our certificate bundle, but, for GDAL, we instead set GDAL_HTTP_UNSAFESSL=YES. Had that option existed, I’m sure we would have used it.

 

That being said, I still feel that, for Windows, using the Certificate Stores is what makes the most sense. That way, in an organizational setting, certificates can be managed via the domain instead of having to configure each workstation separately. That would involve building libcurl with SChannel support instead of OpenSSL. From I can tell, that would only work for Windows XP onwards.

 

André

 

 

From: gdal-dev [mailto:[hidden email]] On Behalf Of Joaquim Luis
Sent: Saturday, June 3, 2017 14:30
To: [hidden email]; Even Rouault <[hidden email]>; Joaquim Luis <[hidden email]>
Subject: Re: [gdal-dev] libcurl and the certificates and Windows

 

For reference

 

 

 

On Sat, 03 Jun 2017 17:22:33 +0100, Even Rouault <[hidden email]> wrote:

On samedi 3 juin 2017 17:04:07 CEST Joaquim Luis wrote:

> Hi,

>

> For quite some time I cannot use the 'vsis' because of certificates issue.

> For example, a GMT test that has a command like this no longer works on

> Windows

>

> gdalinfo

> /vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbit

> .jpg

>

> because

>

> ERROR 11: HTTP response code: 301 - SSL certificate problem: unable to get

> local issuer certificate

> gdalinfo failed - unable to open

> '/vsicurl/http://larryfire.files.wordpress.com/2009/07/untooned_jessicarabbi

> t.jpg'.

>

> It used to work but probably with an older libcurl dll.

> The above is with my own build gdal and dependencies (libcurl included)

> but the same happens with the gisinternals binaries.

>

> I have re(and re)ad this page about the certificates

>

> https://curl.haxx.se/docs/sslcerts.html

>

> but regarding Windows and the curl-ca-bundle.crt file what is said about

> it simply does not work. The only thing that works is setting the ENV

> variable

>

> set CURL_CA_BUNDLE=V:\bin\curl-ca-bundle.crt

>

> Now, we had this in GMT recently and I used the nuke option

>

> curl_easy_setopt (Curl, CURLOPT_SSL_VERIFYPEER, 0L); /* Tell libcurl to

> not verify the peer */

>

> so tried to do the same thing in the GDAL code (the obvious point seamed

> to be VSICurlSetOptions in cpl_vsi_curl.cpp) but still does not work.

 

Someone reported to me a similar issue with recent OSGeo4W.

 

Did you try setting GDAL_HTTP_UNSAFESSL=YES? This is taken into account in CPLHTTPSetOptions() that is called by VSICurlSetOptions(), and this set CURLOPT_SSL_VERIFYPEER=0 and CURLOPT_SSL_VERIFYHOST=0.

 

This solved the issue.

 

 

Thanks, yes that works too (and, no I hadn't tried it before) although it's a different solution than setting  CURL_CA_BUNDLE , which does not turn out the certificates verification. 




_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Even Rouault-2

On lundi 5 juin 2017 19:07:24 CEST Vautour, André (INT) wrote:

> I'd like to add that I think an option like GDAL_HTTP_CA_CERT_FILE or

> GDAL_HTTP_CA_CERT_PATH would be useful to have.

 

See http://gdal.org/cpl__http_8h.html#aee8368b7821300f4b81ef4da8a9c6a29

 

"""

CAINFO=/path/to/bundle.crt. This is path to Certificate Authority (CA) bundle file. By default, it will be looked in a system location. If the CAINFO options is not defined, GDAL will also look if the CURL_CA_BUNDLE environment variable is defined to use it as the CAINFO value, and as a fallback to the SSL_CERT_FILE environment variable. (GDAL >= 2.1.3)

"""

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Joaquim Luis
Even ,

I think you may be interested to see the all extend of https://github.com/curl/curl/issues/1538 Not friendliest place to report (possible) issues.

Joaquim


On lundi 5 juin 2017 19:07:24 CEST Vautour, André (INT) wrote:

> I'd like to add that I think an option like GDAL_HTTP_CA_CERT_FILE or

> GDAL_HTTP_CA_CERT_PATH would be useful to have.

 

See http://gdal.org/cpl__http_8h.html#aee8368b7821300f4b81ef4da8a9c6a29

 

"""

CAINFO=/path/to/bundle.crt. This is path to Certificate Authority (CA) bundle file. By default, it will be looked in a system location. If the CAINFO options is not defined, GDAL will also look if the CURL_CA_BUNDLE environment variable is defined to use it as the CAINFO value, and as a fallback to the SSL_CERT_FILE environment variable. (GDAL >= 2.1.3)

"""

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com





_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: libcurl and the certificates and Windows

Even Rouault-2

On lundi 5 juin 2017 23:02:54 CEST Joaquim Luis wrote:

> https://github.com/curl/curl/issues/1538

 

Hopefully https://trac.osgeo.org/gdal/changeset/38903 should improve the situation a bit (note: completely untested), provided curl-ca-bundle.crt is in one of the searched paths. Note: as far as I can see, this is not the case in gisinternals since it is not in the path where GDAL binaries are...

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev