[MapProxy] database-update triggered reseeding

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[MapProxy] database-update triggered reseeding

Steven Ottens
Hi all,

I'm designing an application where highschool students will go into the
field collect data on the type of lights used in houses. (Quite a cool
project using a home made spectrometer to distinguish between
incandescent, LED and CFL). The idea is that in the last week of October
20000 students will collect this data aggregated on streetlevel and the
results will be rendered on a map.

Everytime a student enters data (s)he obviously wants to see his/her
result ASAP on the map. This means that for a week there will be lots of
updates on the map and at the same time lots of requests. So I was
thinking it would be nice to have an OSM-style setup: every update on
the database triggers a 'dirty' flag on the affected tiles and they are
rendered ASAP.

The usecase is a bit easier than OSM: we will divide the country in a
series of small polygons at which level the data will aggregated as such
only the attribute of the polygon will change, never the geometry and we
could calculate before hand which tiles are be affected by a change in
the database.

Has someone tried to build such a thing with MapProxy before? Any
thoughts on implementation?

Cheers,
Steven


_______________________________________________
MapProxy mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapproxy
Reply | Threaded
Open this post in threaded view
|

Re: [MapProxy] database-update triggered reseeding

Oliver Tonnhofer-3
Hi Steven,

caching and highly dynamic data is tricky :)
You can use a PostGIS query or an HTTP GeoJSON service as the source for a seed/cleanup coverage. But you need to check if that works for your case. From the docs:

Be careful when cleaning up caches with large coverages and levels with lots of tiles (>14). Without coverages, the seed tool works on the file system level and it only needs to check for existing tiles if they should be removed. With coverages, the seed tool traverses the whole tile pyramid and needs to check every posible tile if it exists and if it should be removed. This is much slower.

You could remove the complete lower levels frequently and use a cleanup with a coverage for the upper levels. Or you could disable caching completely and use a simple caching HTTP proxy in front with short expire time. (If the rendering is somewhat fast.)


Regards,
Oliver

--
Oliver Tonnhofer  | Omniscale GmbH & Co KG  | http://omniscale.com
OpenStreetMap WMS and tile services         | http://maps.omniscale.com


> On 28.01.2015, at 15:09, Steven M. Ottens <[hidden email]> wrote:
>
> Hi all,
>
> I'm designing an application where highschool students will go into the field collect data on the type of lights used in houses. (Quite a cool project using a home made spectrometer to distinguish between incandescent, LED and CFL). The idea is that in the last week of October 20000 students will collect this data aggregated on streetlevel and the results will be rendered on a map.
>
> Everytime a student enters data (s)he obviously wants to see his/her result ASAP on the map. This means that for a week there will be lots of updates on the map and at the same time lots of requests. So I was thinking it would be nice to have an OSM-style setup: every update on the database triggers a 'dirty' flag on the affected tiles and they are rendered ASAP.
>
> The usecase is a bit easier than OSM: we will divide the country in a series of small polygons at which level the data will aggregated as such only the attribute of the polygon will change, never the geometry and we could calculate before hand which tiles are be affected by a change in the database.
>
> Has someone tried to build such a thing with MapProxy before? Any thoughts on implementation?
>
> Cheers,
> Steven
>
>
> _______________________________________________
> MapProxy mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/mapproxy


_______________________________________________
MapProxy mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapproxy