MetaTiling - does projection affect?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

MetaTiling - does projection affect?

Bill Teluk
Hi, I've got TileCache 2.10 installed (part of a FGS 9.5 install), with metaTiling turned on for one of my layers.  The cache is certainly working but I don't think that metaTiling is working as I expect - I'm testing if metaTiling is working by checking the number of tiles against the number of ../tilecache/.. requests in the Apache access log - if metaTile is not specified I have 184 requests with 184 tileimages found in the cache, with metaTile=true, I have 184 requests and 316 tile images - shouldn't it be the other way around - less requests for the same number of tiles?  (The boundingbox sizes in the WMS calls are the same size also - checked across several different zoom levels, .)

I'm restarting Apache in between config changes.

I don't have Spherical Mercator as my projection (using GDA94 as the projection/datum etc) - would that make a difference?
My config looks like:

[BaseDetailed]
type=WMS
url=http://pudl-gis-01.networks.in.telstra.com.au:8080/cgi-bin/mapserv?map=/home/maptools/fgs/mapconfig/sbbt/saturndev.map
mapfile=/home/maptools/fgs/mapconfig/sbbt/saturndev.map
extension=png
layers=BorderAndCoast,SettledLand,RecreationalLand,IndustrialLand,Parks,BaseRoads,MetropolitanSuburb,RegionalLocality,PostCodeArea,TownNames,MajorTowns,CapitalCities
bbox=100,-50,155,-5
srs=EPSG:4283
levels=17
resolutions=0.28,0.112,0.056,0.028,0.0112,0.0056,0.0028,0.00112,0.00056,0.00028,0.000112,0.000056,0.000028,0.000014,0.0000112,0.0000056,0.0000028
metaTile=yes
metaBuffer=40

Thanks in advance!
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

Bill Teluk
Correction to item posted to mailing list:
metaTile "off"
184 WMS requests with 184 tileimages found in the cache
metaTile=true
184 WMS requests and 316 tile images
(I assume it should be same number of tile images in both cases, but less WMS requests?)
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

Guillaume Sueur-2
Hi Bill,

The behaviour is exactly as expected here. Metatiling in mainly designed
to seed your cache quicker, because rendering a 2560 x 2560 image and
splitting it into 100 pieces is much faster than rendering 100 tiles.
But it is clearly not designed to be used in a "production" context. The
number of request sent by the client is the same (because the client
doesn't know about metatiling), but for each tile of 256x256 which
doesn't exist in the cache you will generate 100 tiles on the server.
My advice : keep the metatile on for seeding the cache (with
tilecache_seed.py), and put it off when publishing the cache throught
Apache. It can also drive you to file locking and other problems.

Regards,

Guillaume

Le 06/05/2010 06:54, Bill Teluk a écrit :
>
> Correction to item posted to mailing list:
> metaTile "off"
> 184 WMS requests with 184 tileimages found in the cache
> metaTile=true
> 184 WMS requests and 316 tile images
> (I assume it should be same number of tile images in both cases, but less
> WMS requests?)
_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

Bill Teluk
Thanks Guillaume,
It all makes sense now - I was wondering how OpenLayers (from the user's browser) would be able to communicate with TileCache (on the server) to request large tiles instead of small ones!  But as you say, if you use the seeder utility, that then makes perfect sense - since it is on the same machine and has direct access to the config file with the metaTile info.  I will take your advice and turn it off for production use.
Regards, Bill Teluk
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

christopher.schmidt
In reply to this post by Guillaume Sueur-2

On May 6, 2010, at 1:04 AM, ext Guillaume Sueur wrote:

> Hi Bill,
>
> The behaviour is exactly as expected here. Metatiling in mainly designed
> to seed your cache quicker, because rendering a 2560 x 2560 image and
> splitting it into 100 pieces is much faster than rendering 100 tiles.
> But it is clearly not designed to be used in a "production" context. The
> number of request sent by the client is the same (because the client
> doesn't know about metatiling), but for each tile of 256x256 which
> doesn't exist in the cache you will generate 100 tiles on the server.
> My advice : keep the metatile on for seeding the cache (with
> tilecache_seed.py), and put it off when publishing the cache throught
> Apache. It can also drive you to file locking and other problems.

Er... to quote The Doctor: 'what? What?!'?

This is completely untrue. TileCache is designed to support having metaTiling on during production use. The reported behavior is confusing to me, and I don't understand it, but that doesn't mean that metaTiling is not designed for production use. In fact, for many types of access, turning on metaTiling in production is all that allows it to work; with very slow servers, you prevent OpenLayers from 'swamping' an external server (by sending many requests at once) because TileCache performs local locking to prevent re-requesting the same area multiple times.

However, one thing that may be affecting this impression is the fact that the most recent release of TileCache has a bug that causes this locking to fail; the fix for that is http://trac.osgeo.org/tilecache/changeset/391 , which you can apply to your own TileCache deployment, and is the primary motivator for a new TileCache release (which is not yet available, but is on my short-term todo list.) If all of your requests were sent at around the same time, or by using OpenLayesr as a client, I'm not surprised by the buggy behavior, unfortunately.

As for the increased number of tiles -- because TileCache will request 5x5 sections, if you load an area that sits inbetween tile boundaries, you will get extra tiles around the outside. It sounds like you loaded a 14 * 13 area; worst case scenario (being smack dab in the middle of two tilesets) would put the number of tiles at 462, so your experience of having it 'double' (by requesting more data aorund the outside that you didn't need yet) isn't surprising.

So, in short:
 1. metaTile in production: good
 2. TileCache 2.10: bad (and all releases back to 2.03, I think.)
 3. Easy to patch
 4. New release relatively soon
 5. Tile numbers seem reasonable

-- Chris_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

erilem
On Thu, May 6, 2010 at 12:27 PM, [hidden email]
<[hidden email]> wrote:
> [...] TileCache performs local locking to prevent re-requesting the same area multiple times.

Quick question: does TileCache use this locking mechanism for meta
tiles only? If so why?

Thanks,

--
Eric Lemoine

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac, Cedex

Tel : 00 33 4 79 44 44 96
Mail : [hidden email]
http://www.camptocamp.com
_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

christopher.schmidt

On May 6, 2010, at 8:18 AM, ext Eric Lemoine wrote:

> On Thu, May 6, 2010 at 12:27 PM, [hidden email]
> <[hidden email]> wrote:
>> [...] TileCache performs local locking to prevent re-requesting the same area multiple times.
>
> Quick question: does TileCache use this locking mechanism for meta
> tiles only? If so why?

Yes, and because locking didn't exist when the single tile mechanism was written. (Also, it creates significantly more disk access with a relatively limited gain on 'fast' tiles, which was the primary test case.)

I wouldn't be against adding locking to the non-metaTiling case, either by default or as an option, though I'd rather have it as an option because of the increased disk usage.

Also, note that locking in the single tile case is only helpful when there are multiple accessors of the same tile at once, while the tile is ungenerated; the metaTile case, this is the default behavior with any tiling client, so the need there is obviously greater. (Requesting 25 metaTiles instead of one, with the corresponding 25x execution time on the server, would really not work well; even when you realize that render time doesn't scale linearly with area, it's still not a great way to go.)

-- Chris

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

christopher.schmidt

On May 6, 2010, at 9:07 AM, ext [hidden email] wrote:

>
> On May 6, 2010, at 8:18 AM, ext Eric Lemoine wrote:
>
>> On Thu, May 6, 2010 at 12:27 PM, [hidden email]
>> <[hidden email]> wrote:
>>> [...] TileCache performs local locking to prevent re-requesting the same area multiple times.
>>
>> Quick question: does TileCache use this locking mechanism for meta
>> tiles only? If so why?
>
> Yes, and because locking didn't exist when the single tile mechanism was written.

Er, this should obviously say "no".

-- Chris

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

Mark Deneen-2
In reply to this post by christopher.schmidt
On 5/6/2010 7:27 AM, [hidden email] wrote:
>
> Er... to quote The Doctor: 'what? What?!'?
>
> This is completely untrue. TileCache is designed to support having metaTiling on during production use. The reported behavior is confusing to me, and I don't understand it, but that doesn't mean that metaTiling is not designed for production use. In fact, for many types of access, turning on metaTiling in production is all that allows it to work; with very slow servers, you prevent OpenLayers from 'swamping' an external server (by sending many requests at once) because TileCache performs local locking to prevent re-requesting the same area multiple times.
>  

> So, in short:
>  1. metaTile in production: good
>  2. TileCache 2.10: bad (and all releases back to 2.03, I think.)
>  3. Easy to patch
>  4. New release relatively soon
>  5. Tile numbers seem reasonable
>
> -- Chris_______________________________________________
>
>  

Chris,

Would you be interested in a patch to the DiskCache which makes it so
that TileCache will not cache any more tiles, other than what is in the
existing cache?  If a tile is missing, the DiskCache will return a
configurable default tile image.

We use this so that we can seed the whole world, and many levels of
North America.  Then, if a user tries to zoom in to an area which we
don't have tiles for, they just get a default tile image.

Unfortunately, now that I see your message, I wrote this patch because
of the behavior I saw without the patch to 2.10.

I have other changes to my DiskCache as well, so I'd have to separate
them before I could send it to you.  My disk cache calls a function
every time a tile is added to the cache -- the file name is inserted
into a queue and another process runs in the background which optimizes
the .png files.  I could make this configurable as well, but we're
stretching my knowledge of python and dynamic invocation.

Mark
_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

christopher.schmidt
> Chris,
>
> Would you be interested in a patch to the DiskCache which makes it so
> that TileCache will not cache any more tiles, other than what is in the
> existing cache?  If a tile is missing, the DiskCache will return a
> configurable default tile image.

Is this related to the above thread? I don't understand; lack of locking
and adding a feature to TileCache seem unrelated to me.

> We use this so that we can seed the whole world, and many levels of
> North America.  Then, if a user tries to zoom in to an area which we
> don't have tiles for, they just get a default tile image.
>
> Unfortunately, now that I see your message, I wrote this patch because
> of the behavior I saw without the patch to 2.10.

I don't understand why this would be the case; I mean, the feature you
wrote isn't solved by a patch that exists in TileCache, so if you want
that feature, you'll need to write a patch.

However, it shouldn't go in the Cache. It should be a Layer option;
missingTile=/path/to/file.png or some such, and then at the layer level,
we can read that when an empty tile is found and return that instead.

A brief look suggests that Layer.render might be the right place to add
such a check.

-- Chris

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

erilem
In reply to this post by christopher.schmidt
On Thu, May 6, 2010 at 2:07 PM,  <[hidden email]> wrote:

>
> On May 6, 2010, at 8:18 AM, ext Eric Lemoine wrote:
>
>> On Thu, May 6, 2010 at 12:27 PM, [hidden email]
>> <[hidden email]> wrote:
>>> [...] TileCache performs local locking to prevent re-requesting the same area multiple times.
>>
>> Quick question: does TileCache use this locking mechanism for meta
>> tiles only? If so why?
>
> Yes, and because locking didn't exist when the single tile mechanism was written. (Also, it creates significantly more disk access with a relatively limited gain on 'fast' tiles, which was the primary test case.)
>
> I wouldn't be against adding locking to the non-metaTiling case, either by default or as an option, though I'd rather have it as an option because of the increased disk usage.
>
> Also, note that locking in the single tile case is only helpful when there are multiple accessors of the same tile at once, while the tile is ungenerated; the metaTile case, this is the default behavior with any tiling client, so the need there is obviously greater. (Requesting 25 metaTiles instead of one, with the corresponding 25x execution time on the server, would really not work well; even when you realize that render time doesn't scale linearly with area, it's still not a great way to go.)


This makes great sense. Thanks,



--
Eric Lemoine

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac, Cedex

Tel : 00 33 4 79 44 44 96
Mail : [hidden email]
http://www.camptocamp.com
_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

Mark Deneen-2
In reply to this post by christopher.schmidt
On 5/6/2010 10:23 AM, [hidden email] wrote:

>> Chris,
>>
>> Would you be interested in a patch to the DiskCache which makes it so
>> that TileCache will not cache any more tiles, other than what is in the
>> existing cache?  If a tile is missing, the DiskCache will return a
>> configurable default tile image.
>>    
> Is this related to the above thread? I don't understand; lack of locking
> and adding a feature to TileCache seem unrelated to me.
>
>  
I had serious issues running TileCache "live" with meta tiling, so I
opted to seed everything which I want to be viewed.  I then needed a way
to provide a default tile.

>> We use this so that we can seed the whole world, and many levels of
>> North America.  Then, if a user tries to zoom in to an area which we
>> don't have tiles for, they just get a default tile image.
>>
>> Unfortunately, now that I see your message, I wrote this patch because
>> of the behavior I saw without the patch to 2.10.
>>    
> I don't understand why this would be the case; I mean, the feature you
> wrote isn't solved by a patch that exists in TileCache, so if you want
> that feature, you'll need to write a patch.
>
> However, it shouldn't go in the Cache. It should be a Layer option;
> missingTile=/path/to/file.png or some such, and then at the layer level,
> we can read that when an empty tile is found and return that instead.
>
> A brief look suggests that Layer.render might be the right place to add
> such a check.
>
>  
I'll look into the Layer.  Thanks.

> -- Chris
>
>  

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: MetaTiling - does projection affect?

Bill Teluk
In reply to this post by christopher.schmidt
Hi Chris,
Thanks for your help - much appreciated!  I'm after all the performance I can get so I was disappointed when it seemed initially that I couldn't make use of metaTiling.  I will put that patch into my installation.

YES - all the requests in my test were sent in a short time frame - I cleared all my caches and executed a sequence of zooms from an OpenLayers-based web app, one immediately after the other (for performance testing.)  I then measured the #tiles, and Apache calls.  In addition, the areas tested WERE across different tile boundaries - ie. there were a series of partial tiles around some or most of the edges (the calls weren't optimised to align the image window with the tile requests, I used an 800x600 window, I probably should use a 768x512 window maybe.)
Regards, Bill