Resilient hosting of GeoNode

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Resilient hosting of GeoNode

Jac Steiner
I'm working with a government that presently has a GeoNode instance installed locally and accessible publicly from the internet. This is fragile due to infrastructure weaknesses on premises (e.g. unreliable power). I'm considering recommending they move to an instance of GeoNode installed in the cloud (perhaps AWS, but a managed solution would be ideal). This instance would be locally mirrored on site using their current hardware. The local mirror is essential redundancy to accommodate scenarios where the internet may go down (e.g. emergency management). The local mirror should remain relatively in-sync with the cloud master, but infrequent synching, perhaps once a day, should sufficiently cover the anticipated usage patterns.

I figure the devil is in the details here, as in a genuine emergency, if the internet is down and changes are being made locally directly on the mirror, once internet is back, the synching strategy with the master suddenly gets complicated.

I'm not able to find examples of this type of setup in the docs or otherwise. As I'm not a sys admin, I may be using the wrong search terms. Can someone point me toward resources that provide further details on implementing this type of strategy and best practices? If anyone has additional relevant insights they can share, I'd greatly appreciate it.

Appreciate your help.


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Resilient hosting of GeoNode

Carol Kraemer-2

Nifty idea, Jac. Thank you for sharing here and I’d be interested to hear the solution as well.

 

From: geonode-users [mailto:[hidden email]] On Behalf Of Jac Steiner
Sent: Thursday, May 24, 2018 5:54 PM
To: geonode-users <[hidden email]>
Subject: [GeoNode-users] Resilient hosting of GeoNode

 

I'm working with a government that presently has a GeoNode instance installed locally and accessible publicly from the internet. This is fragile due to infrastructure weaknesses on premises (e.g. unreliable power). I'm considering recommending they move to an instance of GeoNode installed in the cloud (perhaps AWS, but a managed solution would be ideal). This instance would be locally mirrored on site using their current hardware. The local mirror is essential redundancy to accommodate scenarios where the internet may go down (e.g. emergency management). The local mirror should remain relatively in-sync with the cloud master, but infrequent synching, perhaps once a day, should sufficiently cover the anticipated usage patterns.

I figure the devil is in the details here, as in a genuine emergency, if the internet is down and changes are being made locally directly on the mirror, once internet is back, the synching strategy with the master suddenly gets complicated.

I'm not able to find examples of this type of setup in the docs or otherwise. As I'm not a sys admin, I may be using the wrong search terms. Can someone point me toward resources that provide further details on implementing this type of strategy and best practices? If anyone has additional relevant insights they can share, I'd greatly appreciate it.

Appreciate your help.


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Resilient hosting of GeoNode

Alessio Fabiani-2
I'd suggest the backup and restore procedure of GeoNode, which dumps almost everything you need and restores it also. (it is worth to do tests first with a dev version, mostly on the restore procedure)

You can use a bash script like this to perform the backups


And a script like this to perform the restore


The settings must be customized for both prod and dev. On the above example I put them into a backup folder, and they look more or less like this


Things to notice on the settings are:

1. GeoServer Data Dir must be accessible and writable by the Tomcat user daemon and the user running the script. Also you need to provide correct path in both cases, prod and dev

2. apps and dumps must be aligned (dumps are just the names of the dump files); you can customize the apps modules as you need, accordingly on what you want to backup/restore




2018-05-25 13:47 GMT+02:00 Carol Kraemer <[hidden email]>:

Nifty idea, Jac. Thank you for sharing here and I’d be interested to hear the solution as well.

 

From: geonode-users [mailto:[hidden email]] On Behalf Of Jac Steiner
Sent: Thursday, May 24, 2018 5:54 PM
To: geonode-users <[hidden email]>
Subject: [GeoNode-users] Resilient hosting of GeoNode

 

I'm working with a government that presently has a GeoNode instance installed locally and accessible publicly from the internet. This is fragile due to infrastructure weaknesses on premises (e.g. unreliable power). I'm considering recommending they move to an instance of GeoNode installed in the cloud (perhaps AWS, but a managed solution would be ideal). This instance would be locally mirrored on site using their current hardware. The local mirror is essential redundancy to accommodate scenarios where the internet may go down (e.g. emergency management). The local mirror should remain relatively in-sync with the cloud master, but infrequent synching, perhaps once a day, should sufficiently cover the anticipated usage patterns.

I figure the devil is in the details here, as in a genuine emergency, if the internet is down and changes are being made locally directly on the mirror, once internet is back, the synching strategy with the master suddenly gets complicated.

I'm not able to find examples of this type of setup in the docs or otherwise. As I'm not a sys admin, I may be using the wrong search terms. Can someone point me toward resources that provide further details on implementing this type of strategy and best practices? If anyone has additional relevant insights they can share, I'd greatly appreciate it.

Appreciate your help.


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==
Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686


http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.


This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Resilient hosting of GeoNode

Paolo Pasquali-2
Hi Alessio,

As I understand the backup/restore procedure works fine in some conditions: good bandwidth, one-way sync, limited amount of data to transfer.

My scenario is similar to Jac's. Due to the poor bandwidth is more efficient to work on a local network GeoNode and then sometimes update the public instance in the cloud (probably on demand).
The public instance can be updated as well by users external to the organization so a one way sync could not be enough.

While we can avoid transferring every time huge raster files (e.g. aerial images, DSM, old scanned maps), pyramids and cache, a mechanism for a two-way sync should be put in place.
What about adopting different workspaces for users internal and external to the organization and by this select the data to transfer. So, I mean always uploading data for a workspace, always downloading for another and syncing the GeoNode db with same criteria.
Do you think it can be feasible?

Or, is working on data differences an option?

Thanks for your help.

Ciao


P.


On Fri, May 25, 2018 at 2:00 PM, Alessio Fabiani <[hidden email]> wrote:
I'd suggest the backup and restore procedure of GeoNode, which dumps almost everything you need and restores it also. (it is worth to do tests first with a dev version, mostly on the restore procedure)

You can use a bash script like this to perform the backups


And a script like this to perform the restore


The settings must be customized for both prod and dev. On the above example I put them into a backup folder, and they look more or less like this


Things to notice on the settings are:

1. GeoServer Data Dir must be accessible and writable by the Tomcat user daemon and the user running the script. Also you need to provide correct path in both cases, prod and dev

2. apps and dumps must be aligned (dumps are just the names of the dump files); you can customize the apps modules as you need, accordingly on what you want to backup/restore




2018-05-25 13:47 GMT+02:00 Carol Kraemer <[hidden email]>:

Nifty idea, Jac. Thank you for sharing here and I’d be interested to hear the solution as well.

 

From: geonode-users [mailto:[hidden email]] On Behalf Of Jac Steiner
Sent: Thursday, May 24, 2018 5:54 PM
To: geonode-users <[hidden email]>
Subject: [GeoNode-users] Resilient hosting of GeoNode

 

I'm working with a government that presently has a GeoNode instance installed locally and accessible publicly from the internet. This is fragile due to infrastructure weaknesses on premises (e.g. unreliable power). I'm considering recommending they move to an instance of GeoNode installed in the cloud (perhaps AWS, but a managed solution would be ideal). This instance would be locally mirrored on site using their current hardware. The local mirror is essential redundancy to accommodate scenarios where the internet may go down (e.g. emergency management). The local mirror should remain relatively in-sync with the cloud master, but infrequent synching, perhaps once a day, should sufficiently cover the anticipated usage patterns.

I figure the devil is in the details here, as in a genuine emergency, if the internet is down and changes are being made locally directly on the mirror, once internet is back, the synching strategy with the master suddenly gets complicated.

I'm not able to find examples of this type of setup in the docs or otherwise. As I'm not a sys admin, I may be using the wrong search terms. Can someone point me toward resources that provide further details on implementing this type of strategy and best practices? If anyone has additional relevant insights they can share, I'd greatly appreciate it.

Appreciate your help.


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==
Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686


http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.


This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users