[MapProxy] mapproxy and S3

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[MapProxy] mapproxy and S3

James Sewell-2
Hi all,

I am using Mapproxy 1.12.0a with a fully seeded tile cache stored on Amazon S3. I'm running Mapproxy on EC2 instances which are using an IAM role.

I have a cache directive as follows:

caches:
 wm_cache:
    format: image/png
    request_format: image/png
    grids: [ "EPSG:3857" ]
    sources: []
    minimize_meta_requests: true
    cache:
      type: s3
      directory: TileCache3857/Carto/
      bucket_name: cache
      directory_layout: arcgis


I have a few questions:

1. How do I attach an on_error section (as seen in a source usually) to this? I don't have a source, but if someone zooms in too far I'd like to set the colour etc..

2. How do I switch IAM role - I'm using cross-account S3 so I can't assign the correct role on instance start.

Thanks for the help,

James Sewell,
Chief Architect



Suite 112, Jones Bay Wharf, 26-32 Pirrama Road, Pyrmont NSW 2009
<a href="tel:(+61)%202%208099%209000" value="+61280999000" style="color:rgb(17,85,204)" target="_blank">(+61) 2 8099 9000  W www.jirotech.com  <a href="tel:(+61)%202%208099%209000" style="color:rgb(17,85,204)" target="_blank">(+61) 2 8099 9099


The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.
_______________________________________________
MapProxy mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapproxy
Reply | Threaded
Open this post in threaded view
|

Re: [MapProxy] mapproxy and S3

Oliver Tonnhofer-3
Hi,

you can create a dummy tile source that references a local file (IIRC `url: file:///path/to/tile`).

For IAM role, please check out the documentation of Boto3. Not sure if it supports what you want to do.


Regards,
Oliver

--
Oliver Tonnhofer  | Omniscale GmbH & Co KG  | https://omniscale.com
OpenStreetMap WMS and tile services         | https://maps.omniscale.com







> On 8. Feb 2018, at 04:49, James Sewell <[hidden email]> wrote:
>
> Hi all,
>
> I am using Mapproxy 1.12.0a with a fully seeded tile cache stored on Amazon S3. I'm running Mapproxy on EC2 instances which are using an IAM role.
>
> I have a cache directive as follows:
>
> caches:
>  wm_cache:
>     format: image/png
>     request_format: image/png
>     grids: [ "EPSG:3857" ]
>     sources: []
>     minimize_meta_requests: true
>     cache:
>       type: s3
>       directory: TileCache3857/Carto/
>       bucket_name: cache
>       directory_layout: arcgis
>
>
> I have a few questions:
>
> 1. How do I attach an on_error section (as seen in a source usually) to this? I don't have a source, but if someone zooms in too far I'd like to set the colour etc..
>
> 2. How do I switch IAM role - I'm using cross-account S3 so I can't assign the correct role on instance start.
>
> Thanks for the help,
>
> James Sewell,
> Chief Architect
>
>
>
> Suite 112, Jones Bay Wharf, 26-32 Pirrama Road, Pyrmont NSW 2009
> P (+61) 2 8099 9000  W www.jirotech.com  F (+61) 2 8099 9099
>
> The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence._______________________________________________
> MapProxy mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/mapproxy

_______________________________________________
MapProxy mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapproxy
Reply | Threaded
Open this post in threaded view
|

Re: [MapProxy] mapproxy and S3

James Sewell-2
Thanks,

Seems the S3 stuff is pretty broken in MapProxy for this use case.

I’ve made some updates which speed up the cache considerably (at the moment it’s a boot session per tile I think) and expose some extra settings (region name, max http pool connections) to make things easier.

I’ll PR next week.

Cheers,
James Sewell 

On Thu, 15 Feb 2018 at 8:22 pm, Oliver Tonnhofer <[hidden email]> wrote:
Hi,

you can create a dummy tile source that references a local file (IIRC `url: file:///path/to/tile`).

For IAM role, please check out the documentation of Boto3. Not sure if it supports what you want to do.


Regards,
Oliver

--
Oliver Tonnhofer  | Omniscale GmbH & Co KG  | https://omniscale.com
OpenStreetMap WMS and tile services         | https://maps.omniscale.com







> On 8. Feb 2018, at 04:49, James Sewell <[hidden email]> wrote:
>
> Hi all,
>
> I am using Mapproxy 1.12.0a with a fully seeded tile cache stored on Amazon S3. I'm running Mapproxy on EC2 instances which are using an IAM role.
>
> I have a cache directive as follows:
>
> caches:
>  wm_cache:
>     format: image/png
>     request_format: image/png
>     grids: [ "EPSG:3857" ]
>     sources: []
>     minimize_meta_requests: true
>     cache:
>       type: s3
>       directory: TileCache3857/Carto/
>       bucket_name: cache
>       directory_layout: arcgis
>
>
> I have a few questions:
>
> 1. How do I attach an on_error section (as seen in a source usually) to this? I don't have a source, but if someone zooms in too far I'd like to set the colour etc..
>
> 2. How do I switch IAM role - I'm using cross-account S3 so I can't assign the correct role on instance start.
>
> Thanks for the help,
>
> James Sewell,
> Chief Architect
>
>
>
> Suite 112, Jones Bay Wharf, 26-32 Pirrama Road, Pyrmont NSW 2009
> P (+61) 2 8099 9000  W www.jirotech.com  F (+61) 2 8099 9099
>
> The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence._______________________________________________
> MapProxy mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/mapproxy

--
James Sewell,
Chief Architect



Suite 112, Jones Bay Wharf, 26-32 Pirrama Road, Pyrmont NSW 2009
<a href="javascript:void(0);" value="+61280999000" style="color:rgb(17,85,204)" target="_blank">(+61) 2 8099 9000  W www.jirotech.com  <a href="javascript:void(0);" style="color:rgb(17,85,204)" target="_blank">(+61) 2 8099 9099


The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.
_______________________________________________
MapProxy mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapproxy