[gdal-dev] vsis3 in TMS ServerUrl?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] vsis3 in TMS ServerUrl?

Sean Gillies-3
Hi all,

I hope you and yours are well in these times.

I'm wondering how close we are to having support for vsis3 URLs in a TMS ServerUrl and if it would be worth doing the work to add it. 

The issue is that HTTPS requests for non-public S3 objects have to be (as I understand it) signed per object. Thus there is no single set of authentication headers we can use for all objects covered by a ServerUrl template like this:

      <ServerUrl><a href="https://my-private-bucket.s3.amazonaws.com/tiles/${z}/${x}/${y}.png">https://my-private-bucket.s3.amazonaws.com/tiles/${z}/${x}/${y}.png?</ServerUrl>

GDAL's vsis3 handler takes care of the signing for single URLs and, syntactically at least, it looks like it would be possible to support the following

      <ServerUrl>/vsis3/my-private-bucket/tiles/${z}/${x}/${y}.png?</ServerUrl>

with the vsis3 handler doing the signing for each individual HTTPS request that is made.

Does this look useful to anyone else? Have any of you found a workaround that is nearly as good?

--
Sean Gillies

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: vsis3 in TMS ServerUrl?

Even Rouault-2
On mercredi 18 mars 2020 18:27:25 CET Sean Gillies wrote:

> Hi all,
>
> I hope you and yours are well in these times.
>
> I'm wondering how close we are to having support for vsis3 URLs in a TMS
> ServerUrl and if it would be worth doing the work to add it.
>
> The issue is that HTTPS requests for non-public S3 objects have to be (as I
> understand it) signed per object. Thus there is no single set of
> authentication headers we can use for all objects covered by a ServerUrl
> template like this:
>
>       <ServerUrl>
> https://my-private-bucket.s3.amazonaws.com/tiles/${z}/${x}/${y}.png
> ?</ServerUrl>
>
> GDAL's vsis3 handler takes care of the signing for single URLs and,
> syntactically at least, it looks like it would be possible to support the
> following
>
>
> <ServerUrl>/vsis3/my-private-bucket/tiles/${z}/${x}/${y}.png?</ServerUrl>
>
> with the vsis3 handler doing the signing for each individual HTTPS request
> that is made.
>
> Does this look useful to anyone else? Have any of you found a workaround
> that is nearly as good?

Sean,

The WMS driver proably pre-dates /vsicurl/ and its children file systems, so
it uses libcurl directly (and, anyway, it uses the curl "multi" API so as to
be able to spawn simultaneous tile downloads, which cannot be done with /
vsicurl/ is the like). But a not-so-hard solution would be to do a little
patching in the WMS driver to use the VSIGetSignedURL() function to transform
a /vsis3/ filename into a signed URL that can be then fed to the existing
download logic of the WMS driver.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev