Considering scrapping the GeoServer mbtiles community module, or making a separate assembly just for the store

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Considering scrapping the GeoServer mbtiles community module, or making a separate assembly just for the store

geowolf
Hi,
I'm looking into publishing the MBTiles vector tiles datastore, and wondering what to do with the existing mbtiles community module
in GeoServer, which is at the moment, in my opinion problematic:
  • It packages into a single zip the store dependency, a WMS output format, and a WPS process.
  • The WMS output format is dangerous for the GeoServer stability, as it allows clients to make a very long request, building a large mbtiles, without any control or timeout.
  • The WPS process adds a dependency to WPS, while maybe someone just wants to display a MBTiles file but without WPS GeoServer won't even start up.
  • The module adds nothing else, there is no custom GUI for the stores or anything like that.
An option can be to create a separate assembly, mbtiles-store-plugin, that only packs the store, and leaving the mbtiles module there.
Works, maybe it's a bit confusing... but I'm wondering, is there anyone caring about the WMS format and the WPS process? 
If not, the easiest thing could be to scrap the module, simplify the assembly, and clean up the docs.

Ideally it could be split into 3 modules, but don't have the time to go there (help to go there is more than welcomed though!).

Opinions?

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.



_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Reply | Threaded
Open this post in threaded view
|

Re: Considering scrapping the GeoServer mbtiles community module, or making a separate assembly just for the store

jody.garnett
Can I ask who the maintainer is? The pom.xml does not say, checking history I see Niels and Justin as some of the initial commits.

I would recommend a separate community module. 

We should keep your feedback in mind (do not pull together so many extensions into a single module). 

On Mon, Nov 25, 2019 at 11:02 AM Andrea Aime <[hidden email]> wrote:
Hi,
I'm looking into publishing the MBTiles vector tiles datastore, and wondering what to do with the existing mbtiles community module
in GeoServer, which is at the moment, in my opinion problematic:
  • It packages into a single zip the store dependency, a WMS output format, and a WPS process.
  • The WMS output format is dangerous for the GeoServer stability, as it allows clients to make a very long request, building a large mbtiles, without any control or timeout.
  • The WPS process adds a dependency to WPS, while maybe someone just wants to display a MBTiles file but without WPS GeoServer won't even start up.
  • The module adds nothing else, there is no custom GUI for the stores or anything like that.
An option can be to create a separate assembly, mbtiles-store-plugin, that only packs the store, and leaving the mbtiles module there.
Works, maybe it's a bit confusing... but I'm wondering, is there anyone caring about the WMS format and the WPS process? 
If not, the easiest thing could be to scrap the module, simplify the assembly, and clean up the docs.

Ideally it could be split into 3 modules, but don't have the time to go there (help to go there is more than welcomed though!).

Opinions?

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
--
--
Jody Garnett


_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Reply | Threaded
Open this post in threaded view
|

Re: Considering scrapping the GeoServer mbtiles community module, or making a separate assembly just for the store

Niels Charlier

This is something from a while back, I did together with Justin for Boundless. I have never used it myself. If anyone would need a code review or has questions about the code in this module, I'm happy to (try to) assist, but I don't really mind what happens to this module. If it has become obsolete it can be removed.

Remark though that the geopkg module depends on it, more specifically for the output format. You would have to move some of the code to this module. Remark however, that all issues you mention apply to this module as well:

- the geopackage contains an outputformat, a WPS process, and dependency on a geotools store.
- I suspect the outputformat would have the same stability issue you mention

Kind Regards

Niels

On 25/11/2019 22:13, Jody Garnett wrote:
Can I ask who the maintainer is? The pom.xml does not say, checking history I see Niels and Justin as some of the initial commits.

I would recommend a separate community module. 

We should keep your feedback in mind (do not pull together so many extensions into a single module). 

On Mon, Nov 25, 2019 at 11:02 AM Andrea Aime <[hidden email]> wrote:
Hi,
I'm looking into publishing the MBTiles vector tiles datastore, and wondering what to do with the existing mbtiles community module
in GeoServer, which is at the moment, in my opinion problematic:
  • It packages into a single zip the store dependency, a WMS output format, and a WPS process.
  • The WMS output format is dangerous for the GeoServer stability, as it allows clients to make a very long request, building a large mbtiles, without any control or timeout.
  • The WPS process adds a dependency to WPS, while maybe someone just wants to display a MBTiles file but without WPS GeoServer won't even start up.
  • The module adds nothing else, there is no custom GUI for the stores or anything like that.
An option can be to create a separate assembly, mbtiles-store-plugin, that only packs the store, and leaving the mbtiles module there.
Works, maybe it's a bit confusing... but I'm wondering, is there anyone caring about the WMS format and the WPS process? 
If not, the easiest thing could be to scrap the module, simplify the assembly, and clean up the docs.

Ideally it could be split into 3 modules, but don't have the time to go there (help to go there is more than welcomed though!).

Opinions?

Cheers
Andrea
==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
--
--
Jody Garnett


_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Reply | Threaded
Open this post in threaded view
|

Re: Considering scrapping the GeoServer mbtiles community module, or making a separate assembly just for the store

geowolf
On Tue, Nov 26, 2019 at 3:11 PM Niels Charlier <[hidden email]> wrote:

This is something from a while back, I did together with Justin for Boundless. I have never used it myself. If anyone would need a code review or has questions about the code in this module, I'm happy to (try to) assist, but I don't really mind what happens to this module. If it has become obsolete it can be removed.

Remark though that the geopkg module depends on it, more specifically for the output format. You would have to move some of the code to this module. Remark however, that all issues you mention apply to this module as well:

- the geopackage contains an outputformat, a WPS process, and dependency on a geotools store.
- I suspect the outputformat would have the same stability issue you mention


Correct, that's why the geopackage store made it into core, but the geopackage module stayed in community
(and the WMS output format should be scrapped from there too, IMHO).

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.



_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Reply | Threaded
Open this post in threaded view
|

Re: Considering scrapping the GeoServer mbtiles community module, or making a separate assembly just for the store

geowolf
In reply to this post by jody.garnett
On Mon, Nov 25, 2019 at 10:13 PM Jody Garnett <[hidden email]> wrote:
Can I ask who the maintainer is? The pom.xml does not say, checking history I see Niels and Justin as some of the initial commits.

I would recommend a separate community module. 

So I've created a separate assembly for the store, updated docs, and added warnings about using the WMS or WPS outputs, 
as they cannot be currently time limited:


Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.



_______________________________________________
Geoserver-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel