TileCache tile storage questions

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

TileCache tile storage questions

Xiaoyu Guan
Dear Tilecache maillinglist:
I have some questions. I coult not find answer on our mailinglist.
I hope someone could answer my questions or any other information which are welcome.

1. Can I store tile in zip or gz format?
For my understand Apache web server can read gz or zip format.

2. tilessize matter? 512*512 vs 256*256 which one is faster
256pixels took too much disk space. if we use 512 pixel, I think tiles totally size will be smaller.
because ,if the 256pixel tile is less than 4kb, the tile still take 4kb disk space. that s why I want to increase to 512*512px.

3. Is MBtiles as same faster as disk tiles, if I store tiles in mbtiles?

4. how much diskspace can MBtiles save?(if we have 500GB disk tiles right now, MBtiles will be 100GB or even smaller?)

Thanks in advance.
Any information are welcome.

Xiaoyu

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

pmgmendes
1. Can I store tile in zip or gz format?
For my understand Apache web server can read gz or zip format. 

The tiles must be stored in a raster format (png, jpg, gif, etc.) support by TileCache. It would not make too much sense to apply compression to image formats (take png as an example) because they are already compressed.
 
2. tilessize matter? 512*512 vs 256*256 which one is faster
256pixels took too much disk space. if we use 512 pixel, I think tiles totally size will be smaller.
because ,if the 256pixel tile is less than 4kb, the tile still take 4kb disk space. that s why I want to increase to 512*512px.

You must evaluate which ones fits you best: Depending on your data complexity, but normally if the tiles are bigger the seeding process will take a longer.

3. Is MBtiles as same faster as disk tiles, if I store tiles in mbtiles? 
4. how much diskspace can MBtiles save?(if we have 500GB disk tiles right now, MBtiles will be 100GB or even smaller?)

TileCache only provides two types of cache DiskCache and MemoryCache. For MBTiles you must go seek other tile cache implementations like MapProxy.


Regards, 
Mendes

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

Xiaoyu Guan
Thanks very much,Mendes.
The idea of store tiles in zip format is saving disk space, right now we have disk space problem. Because a lot of tiles are too small,even smaller than one file cluster. If we can store tiles as one file, that would save a lot of disk space. 

I read tilesize doc v2.1. In the doc, it says tilecache support mbtiles and amazon cloud storage.

Grt

Xiaoyu



Op 16 jan. 2012 om 14:03 heeft Pedro Mendes <[hidden email]> het volgende geschreven:

1. Can I store tile in zip or gz format?
For my understand Apache web server can read gz or zip format. 

The tiles must be stored in a raster format (png, jpg, gif, etc.) support by TileCache. It would not make too much sense to apply compression to image formats (take png as an example) because they are already compressed.
 
2. tilessize matter? 512*512 vs 256*256 which one is faster
256pixels took too much disk space. if we use 512 pixel, I think tiles totally size will be smaller.
because ,if the 256pixel tile is less than 4kb, the tile still take 4kb disk space. that s why I want to increase to 512*512px.

You must evaluate which ones fits you best: Depending on your data complexity, but normally if the tiles are bigger the seeding process will take a longer.

3. Is MBtiles as same faster as disk tiles, if I store tiles in mbtiles? 
4. how much diskspace can MBtiles save?(if we have 500GB disk tiles right now, MBtiles will be 100GB or even smaller?)

TileCache only provides two types of cache DiskCache and MemoryCache. For MBTiles you must go seek other tile cache implementations like MapProxy.


Regards, 
Mendes

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

ritesh ambastha
In reply to this post by pmgmendes
1. You can use mogrify to play with tile compression.
2. 256 x 256 is always a better option
3. Don't know :(

On Mon, Jan 16, 2012 at 6:33 PM, Pedro Mendes <[hidden email]> wrote:
1. Can I store tile in zip or gz format?
For my understand Apache web server can read gz or zip format. 

The tiles must be stored in a raster format (png, jpg, gif, etc.) support by TileCache. It would not make too much sense to apply compression to image formats (take png as an example) because they are already compressed.
 
2. tilessize matter? 512*512 vs 256*256 which one is faster
256pixels took too much disk space. if we use 512 pixel, I think tiles totally size will be smaller.
because ,if the 256pixel tile is less than 4kb, the tile still take 4kb disk space. that s why I want to increase to 512*512px.

You must evaluate which ones fits you best: Depending on your data complexity, but normally if the tiles are bigger the seeding process will take a longer.

3. Is MBtiles as same faster as disk tiles, if I store tiles in mbtiles? 
4. how much diskspace can MBtiles save?(if we have 500GB disk tiles right now, MBtiles will be 100GB or even smaller?)

TileCache only provides two types of cache DiskCache and MemoryCache. For MBTiles you must go seek other tile cache implementations like MapProxy.


Regards, 
Mendes

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache



_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

pmgmendes
In reply to this post by Xiaoyu Guan
I read tilesize doc v2.1. In the doc, it says tilecache support mbtiles and amazon cloud storage.

You're right. Sorry, my bad. A cache persistent in MBTiles will not scale as well as other options. Theoretically, a sqlite db could be terabyte-sized (http://www.sqlite.org/features.html) but keep in mind that the access to the tiles would degraded just after a few gigabytes. I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings.

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

Xiaoyu Guan
Thanks for quick reply. It would be nice that someone can update the doc. And who can answer the following question. Otherwise, I have to dig into tilecache source code ...

" I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings."


Grt


Xiaoyu


Op 17 jan. 2012 om 11:12 heeft Pedro Mendes <[hidden email]> het volgende geschreven:

I read tilesize doc v2.1. In the doc, it says tilecache support mbtiles and amazon cloud storage.

You're right. Sorry, my bad. A cache persistent in MBTiles will not scale as well as other options. Theoretically, a sqlite db could be terabyte-sized (http://www.sqlite.org/features.html) but keep in mind that the access to the tiles would degraded just after a few gigabytes. I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings.

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

nt2008
2. You can save your HDD by changing python code to storage only tiles which have data.

On Wed, Jan 18, 2012 at 3:46 AM, Xiaoyuguan <[hidden email]> wrote:
Thanks for quick reply. It would be nice that someone can update the doc. And who can answer the following question. Otherwise, I have to dig into tilecache source code ...


" I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings."


Grt


Xiaoyu


Op 17 jan. 2012 om 11:12 heeft Pedro Mendes <[hidden email]> het volgende geschreven:

I read tilesize doc v2.1. In the doc, it says tilecache support mbtiles and amazon cloud storage.

You're right. Sorry, my bad. A cache persistent in MBTiles will not scale as well as other options. Theoretically, a sqlite db could be terabyte-sized (http://www.sqlite.org/features.html) but keep in mind that the access to the tiles would degraded just after a few gigabytes. I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings.

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache



_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

Xiaoyu Guan
2. You can save your HDD by changing python code to storage only tiles which have data.

Good idea. What will happen, if user do request to an extent which is no data or sea? Will tilecache show pink tiles?

Grt

Xiaoyu



Op 18 jan. 2012 om 03:25 heeft Nguyễn Hoàng Nam <[hidden email]> het volgende geschreven:

2. You can save your HDD by changing python code to storage only tiles which have data.

On Wed, Jan 18, 2012 at 3:46 AM, Xiaoyuguan <[hidden email]> wrote:
Thanks for quick reply. It would be nice that someone can update the doc. And who can answer the following question. Otherwise, I have to dig into tilecache source code ...


" I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings."


Grt


Xiaoyu


Op 17 jan. 2012 om 11:12 heeft Pedro Mendes <[hidden email]> het volgende geschreven:

I read tilesize doc v2.1. In the doc, it says tilecache support mbtiles and amazon cloud storage.

You're right. Sorry, my bad. A cache persistent in MBTiles will not scale as well as other options. Theoretically, a sqlite db could be terabyte-sized (http://www.sqlite.org/features.html) but keep in mind that the access to the tiles would degraded just after a few gigabytes. I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings.

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache



_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache
Reply | Threaded
Open this post in threaded view
|

Re: TileCache tile storage questions

nt2008
You should replace default pink tile by a blank tile
Here is my example. The size of folder to storage tiles on disk is only:
 3Gb --> http://112.213.86.71/irrigation
100 Mb --> http://112.213.86.71/dienbien

We also apply this principle for our website: http://imap.com.vn/map

2012/1/18 Xiaoyuguan <[hidden email]>
2. You can save your HDD by changing python code to storage only tiles which have data.

Good idea. What will happen, if user do request to an extent which is no data or sea? Will tilecache show pink tiles?

Grt

Xiaoyu



Op 18 jan. 2012 om 03:25 heeft Nguyễn Hoàng Nam <[hidden email]> het volgende geschreven:

2. You can save your HDD by changing python code to storage only tiles which have data.

On Wed, Jan 18, 2012 at 3:46 AM, Xiaoyuguan <[hidden email]> wrote:
Thanks for quick reply. It would be nice that someone can update the doc. And who can answer the following question. Otherwise, I have to dig into tilecache source code ...


" I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings."


Grt


Xiaoyu


Op 17 jan. 2012 om 11:12 heeft Pedro Mendes <[hidden email]> het volgende geschreven:

I read tilesize doc v2.1. In the doc, it says tilecache support mbtiles and amazon cloud storage.

You're right. Sorry, my bad. A cache persistent in MBTiles will not scale as well as other options. Theoretically, a sqlite db could be terabyte-sized (http://www.sqlite.org/features.html) but keep in mind that the access to the tiles would degraded just after a few gigabytes. I don't know if TileCache follows the same strategy as MapProxy regarding tiles with the same RGB (i.e. lake areas) as it reuses tiles permitting huge disk space savings.

_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache




_______________________________________________
Tilecache mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/tilecache