Poor performance GeoNode

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Poor performance GeoNode

Ana Boljevic
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg). 
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

-- 

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users

geoserver.png (98K) Download Attachment
free.JPG (174K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Angelos Tzotsos
You need to add more workers on your WSGI server.

Best,
Angelos

On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

John Jediny
apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Ana Boljevic
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Alessio Fabiani-2
Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Ana Boljevic
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users

novi_DV_35kV_Milocer_Buljarica.kml (248K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Alessio Fabiani-2
From the video it looks like the issue is on GeoServer side.

There could be many reasons for that, starting from the type of data (in the example I see you are asking vectorial datasets), how they are stored (shapefiles or DB), connections available and memory of the DB, complexity of styles (in this case look quite simple, unless you have a lot of rules), cache enabled or not (in this case it seems like the cache is disabled).

If you feel comfortable you can contact me privately and share with me passwords for the instance so I can take a first look at the settings. From the informations I have right I cannot say much more than this currently.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Fri, Sep 8, 2017 at 9:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: <a href="tel:0584%20962313" value="+390584962313" target="_blank">+39 0584 962313
fax:     <a href="tel:0584%20166%200272" value="+3905841660272" target="_blank">+39 0584 1660272
mob:   <a href="tel:331%20623%203686" value="+393316233686" target="_blank">+39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Alessio Fabiani-2
... adding to the previous email, it is also possible that somehow the HTTP server and/or network configuration cause a delay. Sometimes requests are slow simply because they are waiting for some timeout to expire. Tipically a DNS or proxy issue.

Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Fri, Sep 8, 2017 at 10:04 AM, Alessio Fabiani <[hidden email]> wrote:
From the video it looks like the issue is on GeoServer side.

There could be many reasons for that, starting from the type of data (in the example I see you are asking vectorial datasets), how they are stored (shapefiles or DB), connections available and memory of the DB, complexity of styles (in this case look quite simple, unless you have a lot of rules), cache enabled or not (in this case it seems like the cache is disabled).

If you feel comfortable you can contact me privately and share with me passwords for the instance so I can take a first look at the settings. From the informations I have right I cannot say much more than this currently.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: <a href="tel:0584%20962313" value="+390584962313" target="_blank">+39 0584 962313
fax:     <a href="tel:0584%20166%200272" value="+3905841660272" target="_blank">+39 0584 1660272
mob:   <a href="tel:331%20623%203686" value="+393316233686" target="_blank">+39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Fri, Sep 8, 2017 at 9:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: <a href="tel:0584%20962313" value="+390584962313" target="_blank">+39 0584 962313
fax:     <a href="tel:0584%20166%200272" value="+3905841660272" target="_blank">+39 0584 1660272
mob:   <a href="tel:331%20623%203686" value="+393316233686" target="_blank">+39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users





_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

John Jediny
In reply to this post by Ana Boljevic
Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: <a href="tel:+39%200584%20962313" value="+390584962313" target="_blank">+39 0584 962313
fax:     <a href="tel:+39%200584%20166%200272" value="+3905841660272" target="_blank">+39 0584 1660272
mob:   <a href="tel:+39%20331%20623%203686" value="+393316233686" target="_blank">+39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: <a href="tel:(973)%20931-0580" value="+19739310580" target="_blank">973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users





--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

jondoig

Hi Ana

 

Great soundtrack! 😃

 

Have you seeded the cache for all your layers?

 

http(s)://<yourHostName>/geoserver/gwc/rest/seed/<layerName>

 

Tip: make this task easier and more prominent by adding the option under Edit Layer for admin users (see commit 38a1931 on my repo):

 

 

Regards

Jonathan

 

From: geonode-users [mailto:[hidden email]] On Behalf Of John Jediny
Sent: Thursday, 14 September 2017 7:45 AM
To: Ana Boljevic <[hidden email]>
Cc: Alessio Fabiani <[hidden email]>; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

 

Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

 

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:

Hi Alessio,


Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.

Regards,


Ana Boljevic 

-- 


From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]


Subject: Re: [GeoNode-users] Poor performance GeoNode

 

Can you please better explain what you mean with "Map Performance" exactly?

 

Getting the list of layers or rendering the map?

 

Also, how many layers you have and which type (vectorial / raster) ?

 

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

 

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.

 

 


Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: <a href="tel:&#43;39%200584%20962313" target="_blank">+39 0584 962313
fax:     <a href="tel:&#43;39%200584%20166%200272" target="_blank">+39 0584 1660272
mob:   <a href="tel:&#43;39%20331%20623%203686" target="_blank">+39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

 

On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:

Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic

-- 

 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

 

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:

You need to add more workers on your WSGI server.

Best,
Angelos



On 08/29/2017 11:43 AM, Ana Boljevic wrote:

Dear Geonode Community,
 
We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database
 
For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.
 
Regards,
 
Ana Boljevic
 
--
 

 

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users

 

-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--

Cell: <a href="tel:(973)%20931-0580" target="_blank">973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users

 




--

Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Ana Boljevic
Dear Jonathan,

We couldn't seed cache for any layer. Every time we receive this message (path is: Caching Defaults-Go to the embeded GeoWebCache home page-A list of all the layers and automatic demos-Seed this layer-Submit):
Unknown or malformed request. Please try again, somtimes the form is not properly received. This frequently happens on the first POST after a restart. The POST was to /geoserver/gwc/rest/seed/cedis:novi_110kV_Andrijevica_Berane
Can you be more specific about your tip, we don't understand it?

Regards,

Ana

On 9/14/2017 7:46 AM, Jonathan Doig wrote:

Hi Ana

 

Great soundtrack! 😃

 

Have you seeded the cache for all your layers?

 

http(s)://<yourHostName>/geoserver/gwc/rest/seed/<layerName>

 

Tip: make this task easier and more prominent by adding the option under Edit Layer for admin users (see commit 38a1931 on my repo):

 

 

Regards

Jonathan

 

From: geonode-users [[hidden email]] On Behalf Of John Jediny
Sent: Thursday, 14 September 2017 7:45 AM
To: Ana Boljevic [hidden email]
Cc: Alessio Fabiani [hidden email]; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

 

Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

 

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:

Hi Alessio,


Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.

Regards,


Ana Boljevic 

-- 


From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]


Subject: Re: [GeoNode-users] Poor performance GeoNode

 

Can you please better explain what you mean with "Map Performance" exactly?

 

Getting the list of layers or rendering the map?

 

Also, how many layers you have and which type (vectorial / raster) ?

 

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

 

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.

 

 


Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: <a href="tel:&#43;39%200584%20962313" target="_blank" moz-do-not-send="true">+39 0584 962313
fax:     <a href="tel:&#43;39%200584%20166%200272" target="_blank" moz-do-not-send="true">+39 0584 1660272
mob:   <a href="tel:&#43;39%20331%20623%203686" target="_blank" moz-do-not-send="true">+39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

 

On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:

Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic

-- 

 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

 

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:

You need to add more workers on your WSGI server.

Best,
Angelos



On 08/29/2017 11:43 AM, Ana Boljevic wrote:

Dear Geonode Community,
 
We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database
 
For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.
 
Regards,
 
Ana Boljevic
 
--
 

 

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users

 

-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--

Cell: <a href="tel:%28973%29%20931-0580" target="_blank" moz-do-not-send="true">973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users

 




--

Cell: 973-931-0580



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Ana Boljevic
In reply to this post by John Jediny
Dear John,

We have in-house server on the corporate LAN (100Mbps). We have tested bandwidth speed with iperf. We have started iperf on 4 Windows clients and on Ubuntu/GeoNode Server. Every time all clients are sharing 100Mbps (cca 25Mbps per client). So, we think it is not the network issue.

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Virtualization platform is Oracle VM.
 In the moment when performance decrease, a server eats only 6-7GB of RAM (free is 17-18GB of RAM) and load average is about 7.

We have enabled GeoWebCache by doing the following:
- Caching to external postgresql database
Tile caching - Disk quota....Enable disk Quota ....Maximum tile cache size 50GiB
Disk quota check frequency:600s
Disk quota store type....External database
Target database type....PostgreSQL
JDBC data source...GeoServer managed connection pool
JDBC Driver class name.....org.postgresql.Driver
JDBC connection URL.....jdbc:postgresql://localhost:5432/geonode
User name..... geonode
Password.... ******
Validation query.....select 1

sudo service tomcat7 stop
sudo service apache2 stop
mkdir /var/cache/geoserver
chmod -R 777 geoserver
In:
vi /usr/share/geoserver/WEB-INF/web.xml
added:
<context-param>
       <param-name>GEOWEBCACHE_CACHE_DIR</param-name>
       <param-value>/var/cache/geoserver/</param-value>
</context-param>
sudo reboot

Here is the link of our map containing 140 internal layers (without external WMS layers) during heavy rendering of 4 clients:
https://www.dropbox.com/s/2ncig40ee0i0ybo/2017-09-14-12-08-17.mp4?dl=0

During online connection of Mr. Alessio Fabiani, he has made following changes:
-Geofence:
Size of the raw cache:600000
Cache refewsh interval: 600000
Cache expire interval: 600000

-Disk Quota:
Disk Quota check frequency : 600
Max.connections: 20

- Store - datastore:
max connections: 50
min connections: 10
fetch size : 100000

-WCS: Overview policy: Closest overview

Regards,

Ana

On 9/13/2017 11:44 PM, John Jediny wrote:
Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Alessio Fabiani

@alfa7691 Founder/Technical Lead

GeoSolutions S.A.S. Via di Montramito 3/A 55054  Massarosa (LU) Italy phone: <a href="tel:&#43;39%200584%20962313" value="&#43;390584962313" target="_blank" moz-do-not-send="true"> +39 0584 962313 fax:     <a href="tel:&#43;39%200584%20166%200272" value="&#43;3905841660272" target="_blank" moz-do-not-send="true">+39 0584 1660272 mob:   <a href="tel:&#43;39%20331%20623%203686" value="&#43;393316233686" target="_blank" moz-do-not-send="true">+39 331 6233686 http://www.geo-solutions.it http://twitter.com/geosolutions_it -------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: <a href="tel:%28973%29%20931-0580" value="&#43;19739310580" target="_blank" moz-do-not-send="true"> 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users





--
Cell: 973-931-0580



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

John Jediny
GeoWebCache sounds like the issue

On Thu, Sep 14, 2017 at 6:29 AM, Ana Boljevic <[hidden email]> wrote:
Dear John,

We have in-house server on the corporate LAN (100Mbps). We have tested bandwidth speed with iperf. We have started iperf on 4 Windows clients and on Ubuntu/GeoNode Server. Every time all clients are sharing 100Mbps (cca 25Mbps per client). So, we think it is not the network issue.

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Virtualization platform is Oracle VM.
 In the moment when performance decrease, a server eats only 6-7GB of RAM (free is 17-18GB of RAM) and load average is about 7.

We have enabled GeoWebCache by doing the following:
- Caching to external postgresql database
Tile caching - Disk quota....Enable disk Quota ....Maximum tile cache size 50GiB
Disk quota check frequency:600s
Disk quota store type....External database
Target database type....PostgreSQL
JDBC data source...GeoServer managed connection pool
JDBC Driver class name.....org.postgresql.Driver
JDBC connection URL.....jdbc:postgresql://localhost:5432/geonode
User name..... geonode
Password.... ******
Validation query.....select 1

sudo service tomcat7 stop
sudo service apache2 stop
mkdir /var/cache/geoserver
chmod -R 777 geoserver
In:
vi /usr/share/geoserver/WEB-INF/web.xml
added:
<context-param>
       <param-name>GEOWEBCACHE_CACHE_DIR</param-name>
       <param-value>/var/cache/geoserver/</param-value>
</context-param>
sudo reboot

Here is the link of our map containing 140 internal layers (without external WMS layers) during heavy rendering of 4 clients:
https://www.dropbox.com/s/2ncig40ee0i0ybo/2017-09-14-12-08-17.mp4?dl=0

During online connection of Mr. Alessio Fabiani, he has made following changes:
-Geofence:
Size of the raw cache:600000
Cache refewsh interval: 600000
Cache expire interval: 600000

-Disk Quota:
Disk Quota check frequency : 600
Max.connections: 20

- Store - datastore:
max connections: 50
min connections: 10
fetch size : 100000

-WCS: Overview policy: Closest overview

Regards,

Ana


On 9/13/2017 11:44 PM, John Jediny wrote:
Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Alessio Fabiani

@alfa7691 Founder/Technical Lead

GeoSolutions S.A.S. Via di Montramito 3/A 55054  Massarosa (LU) Italy phone: <a href="tel:+39%200584%20962313" value="+390584962313" target="_blank"> +39 0584 962313 fax:     <a href="tel:+39%200584%20166%200272" value="+3905841660272" target="_blank">+39 0584 1660272 mob:   <a href="tel:+39%20331%20623%203686" value="+393316233686" target="_blank">+39 331 6233686 http://www.geo-solutions.it http://twitter.com/geosolutions_it -------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: <a href="tel:%28973%29%20931-0580" value="+19739310580" target="_blank"> 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users





--
Cell: <a href="tel:(973)%20931-0580" value="+19739310580" target="_blank">973-931-0580





--
Cell: 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Alessio Fabiani-2
We did tests with JMeter directly to GeoServer with 20 parallel threads. GeoServer is not the issue.

My guess is that on the map there are some layers (external or remote or Layer Groups containing remote layers or something) that cause that slowness.

Regards,

Alessio Fabiani

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead


GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Fri, Sep 15, 2017 at 12:34 AM, John Jediny <[hidden email]> wrote:
GeoWebCache sounds like the issue

On Thu, Sep 14, 2017 at 6:29 AM, Ana Boljevic <[hidden email]> wrote:
Dear John,

We have in-house server on the corporate LAN (100Mbps). We have tested bandwidth speed with iperf. We have started iperf on 4 Windows clients and on Ubuntu/GeoNode Server. Every time all clients are sharing 100Mbps (cca 25Mbps per client). So, we think it is not the network issue.

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Virtualization platform is Oracle VM.
 In the moment when performance decrease, a server eats only 6-7GB of RAM (free is 17-18GB of RAM) and load average is about 7.

We have enabled GeoWebCache by doing the following:
- Caching to external postgresql database
Tile caching - Disk quota....Enable disk Quota ....Maximum tile cache size 50GiB
Disk quota check frequency:600s
Disk quota store type....External database
Target database type....PostgreSQL
JDBC data source...GeoServer managed connection pool
JDBC Driver class name.....org.postgresql.Driver
JDBC connection URL.....jdbc:postgresql://localhost:5432/geonode
User name..... geonode
Password.... ******
Validation query.....select 1

sudo service tomcat7 stop
sudo service apache2 stop
mkdir /var/cache/geoserver
chmod -R 777 geoserver
In:
vi /usr/share/geoserver/WEB-INF/web.xml
added:
<context-param>
       <param-name>GEOWEBCACHE_CACHE_DIR</param-name>
       <param-value>/var/cache/geoserver/</param-value>
</context-param>
sudo reboot

Here is the link of our map containing 140 internal layers (without external WMS layers) during heavy rendering of 4 clients:
https://www.dropbox.com/s/2ncig40ee0i0ybo/2017-09-14-12-08-17.mp4?dl=0

During online connection of Mr. Alessio Fabiani, he has made following changes:
-Geofence:
Size of the raw cache:600000
Cache refewsh interval: 600000
Cache expire interval: 600000

-Disk Quota:
Disk Quota check frequency : 600
Max.connections: 20

- Store - datastore:
max connections: 50
min connections: 10
fetch size : 100000

-WCS: Overview policy: Closest overview

Regards,

Ana


On 9/13/2017 11:44 PM, John Jediny wrote:
Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Alessio Fabiani

@alfa7691 Founder/Technical Lead

GeoSolutions S.A.S. Via di Montramito 3/A 55054  Massarosa (LU) Italy phone: <a href="tel:+39%200584%20962313" value="+390584962313" target="_blank"> +39 0584 962313 fax:     <a href="tel:+39%200584%20166%200272" value="+3905841660272" target="_blank">+39 0584 1660272 mob:   <a href="tel:+39%20331%20623%203686" value="+393316233686" target="_blank">+39 331 6233686 http://www.geo-solutions.it http://twitter.com/geosolutions_it -------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: <a href="tel:%28973%29%20931-0580" value="+19739310580" target="_blank"> 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users





--
Cell: <a href="tel:(973)%20931-0580" value="+19739310580" target="_blank">973-931-0580





--
Cell: 973-931-0580



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users
Reply | Threaded
Open this post in threaded view
|

Re: Poor performance GeoNode

Ana Boljevic
Dear Alessio,

Last video is made with map that contains 3 layer groups (made in Geoserver).
https://www.dropbox.com/s/2ncig40ee0i0ybo/2017-09-14-12-08-17.mp4?dl=0
Those three Group layers are consist od 40-50 LOCAL layers each. Typical kml file that represent one of those layers has been sent to you previously. 
Only external component is Google Satellite (Basemap)....It doesn't cause slowness.

Regards,

Ana


On 9/15/2017 10:40 AM, Alessio Fabiani wrote:
We did tests with JMeter directly to GeoServer with 20 parallel threads. GeoServer is not the issue.

My guess is that on the map there are some layers (external or remote or Layer Groups containing remote layers or something) that cause that slowness.

Regards,

Alessio Fabiani

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Alessio Fabiani

@alfa7691 Founder/Technical Lead

GeoSolutions S.A.S. Via di Montramito 3/A 55054  Massarosa (LU) Italy phone: +39 0584 962313 fax:     +39 0584 1660272 mob:   +39 331 6233686 http://www.geo-solutions.it http://twitter.com/geosolutions_it -------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Fri, Sep 15, 2017 at 12:34 AM, John Jediny <[hidden email]> wrote:
GeoWebCache sounds like the issue

On Thu, Sep 14, 2017 at 6:29 AM, Ana Boljevic <[hidden email]> wrote:
Dear John,

We have in-house server on the corporate LAN (100Mbps). We have tested bandwidth speed with iperf. We have started iperf on 4 Windows clients and on Ubuntu/GeoNode Server. Every time all clients are sharing 100Mbps (cca 25Mbps per client). So, we think it is not the network issue.

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Virtualization platform is Oracle VM.
 In the moment when performance decrease, a server eats only 6-7GB of RAM (free is 17-18GB of RAM) and load average is about 7.

We have enabled GeoWebCache by doing the following:
- Caching to external postgresql database
Tile caching - Disk quota....Enable disk Quota ....Maximum tile cache size 50GiB
Disk quota check frequency:600s
Disk quota store type....External database
Target database type....PostgreSQL
JDBC data source...GeoServer managed connection pool
JDBC Driver class name.....org.postgresql.Driver
JDBC connection URL.....jdbc:postgresql://localhost:5432/geonode
User name..... geonode
Password.... ******
Validation query.....select 1

sudo service tomcat7 stop
sudo service apache2 stop
mkdir /var/cache/geoserver
chmod -R 777 geoserver
In:
vi /usr/share/geoserver/WEB-INF/web.xml
added:
<context-param>
       <param-name>GEOWEBCACHE_CACHE_DIR</param-name>
       <param-value>/var/cache/geoserver/</param-value>
</context-param>
sudo reboot

Here is the link of our map containing 140 internal layers (without external WMS layers) during heavy rendering of 4 clients:
https://www.dropbox.com/s/2ncig40ee0i0ybo/2017-09-14-12-08-17.mp4?dl=0

During online connection of Mr. Alessio Fabiani, he has made following changes:
-Geofence:
Size of the raw cache:600000
Cache refewsh interval: 600000
Cache expire interval: 600000

-Disk Quota:
Disk Quota check frequency : 600
Max.connections: 20

- Store - datastore:
max connections: 50
min connections: 10
fetch size : 100000

-WCS: Overview policy: Closest overview

Regards,

Ana


On 9/13/2017 11:44 PM, John Jediny wrote:
Are you running it on a cloud service provider or in-house? this seems like a network latency issue at first watch. Can you share the specs of the operating systems/machines (e.g. VM/containers) being used for total CPU/RAM/Network?

On Fri, Sep 8, 2017 at 3:11 AM, Ana Boljevic <[hidden email]> wrote:
Hi Alessio,

Please receive screen video recording link, so you can see by yourself what do we mean by "Poor Performance". 
https://www.dropbox.com/s/2ai2n8764p1ny9n/2017-09-08-08-38-33.flv?dl=0
This recording was taken during of heavy map rendering of 4 clients. We have 144 vector layers and 2 external WMS layers (behaviour is the same when we don't use WMS layers. In video link, we have used only local vector layers, not external WMS layers). We are not experienced enough to know what does cause the issue (GeoNode or GeoServer).
Please reveive one typical kml layer in the attachment.


Regards,

Ana Boljevic 
-- 

From: Alessio Fabiani [[hidden email]]
Sent: Thursday, September 07, 2017 4:21 PM
To: Ana Boljevic
Cc: John Jediny; Angelos Tzotsos; [hidden email]

Subject: Re: [GeoNode-users] Poor performance GeoNode

Can you please better explain what you mean with "Map Performance" exactly?

Getting the list of layers or rendering the map?

Also, how many layers you have and which type (vectorial / raster) ?

Are you able to identify performance issues? Those are on GeoNode or GeoServer?

In the first case it is certainly an issue of your HTTP Server configuration (and most probably the proxy module), in the second case it could be an issue of data not correctly processed.



Regards,

Alessio Fabiani

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Alessio Fabiani

@alfa7691 Founder/Technical Lead

GeoSolutions S.A.S. <a href="https://maps.google.com/?q=Via&#43;di&#43;Montramito&#43;3/A%0D&#43;55054&#43;%C2%A0Massarosa&amp;entry=gmail&amp;source=g" target="_blank" moz-do-not-send="true">Via di Montramito 3/A <a href="https://maps.google.com/?q=Via&#43;di&#43;Montramito&#43;3/A%0D&#43;55054&#43;%C2%A0Massarosa&amp;entry=gmail&amp;source=g" target="_blank" moz-do-not-send="true">55054 <a href="https://maps.google.com/?q=Via&#43;di&#43;Montramito&#43;3/A%0D&#43;55054&#43;%3Chttps://maps.google.com/?q%3DVia%2Bdi%2BMontramito%2B3/A%250D%2B55054%2B%25C2%25A0Massarosa%26entry%3Dgmail%26source%3Dg%3E%C2%A0Massarosa&#43;%3Chttps://maps.google.com/?q%3DVia%2Bdi%2BMontramito%2B3/A%250D%2B55054%2B%25C2%25A0Massarosa%26entry%3Dgmail%26source%3Dg%3E&amp;entry=gmail&amp;source=g" moz-do-not-send="true"> <a href="https://maps.google.com/?q=Via&#43;di&#43;Montramito&#43;3/A%0D&#43;55054&#43;%C2%A0Massarosa&amp;entry=gmail&amp;source=g" target="_blank" moz-do-not-send="true">Massarosa (LU) Italy phone: <a href="tel:&#43;39%200584%20962313" value="&#43;390584962313" target="_blank" moz-do-not-send="true"> +39 0584 962313 fax:     <a href="tel:&#43;39%200584%20166%200272" value="&#43;3905841660272" target="_blank" moz-do-not-send="true">+39 0584 1660272 mob:   <a href="tel:&#43;39%20331%20623%203686" value="&#43;393316233686" target="_blank" moz-do-not-send="true">+39 331 6233686 http://www.geo-solutions.it http://twitter.com/geosolutions_it -------------------------------------------------------

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Thu, Sep 7, 2017 at 8:54 AM, Ana Boljevic <[hidden email]> wrote:
Hi Angelos and John,
Let me explain one more time, our performance issue. We have done fresh Geonode 2.6.2 installation (standard deployment), give VM 24GB RAM and 12 CPU. After installation, we have done:
1) Replace OpenJVM with OracleJVM 
2) We've increased the JVM memory by adding into /etc/default/tomcat7 
JAVA_OPTS="-Xmx12288M -Xms12288M -XX:MaxPermSize=1024M -XX:CompileCommand=exclude,net/sf/saxon/event/ReceivingContentHandler.startEvent"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

Additionally, we have tried the following:
1) In /etc/tomcat7/server.xml added maxThreads="50":
 
  <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" maxThreads="50"/>
2) In /etc/apache2/mods-enabled/mpm_event.conf

try to change conf parameters with no results:
<IfModule mpm_event_module>
        StartServers                     2
        MinSpareThreads          25
        MaxSpareThreads          75
        ThreadLimit                      64
        ThreadsPerChild          25
        MaxRequestWorkers         150
        MaxConnectionsPerChild   0
</IfModule>

Still, when 4-5 clients try to render maps, performance radically decrease and geonode become unresponsive.
Do you have any idea how to increse performance ?
Best regards,
Ana Boljevic
-- 


From: John Jediny [[hidden email]]
Sent: Saturday, September 02, 2017 3:32 AM
To: Angelos Tzotsos
Cc: Ana Boljevic; [hidden email]
Subject: Re: [GeoNode-users] Poor performance GeoNode

apache2 mod_wsgi needs tuning. Add workers and up memory limits per worker

On Fri, Sep 1, 2017 at 3:56 PM, Angelos Tzotsos <[hidden email]> wrote:
You need to add more workers on your WSGI server.

Best,
Angelos


On 08/29/2017 11:43 AM, Ana Boljevic wrote:
Dear Geonode Community,

We have installed fresh Geonode v2.6.2 on Ubuntu Server 16.04. In order to speed up system, we gave Geonode 24GB of RAM and 12 CPU. Unfortulnally, performance are still poor. Apart from basic installation (standard, not developement deployment) we have done:
1) Replace OpenJVM with OracleJVM (see attachment geoserver.png)
2) We've increased the JVM memory by adding into /etc/default/tomcat7
JAVA_OPTS="-Djava.awt.headless=true -Xms6144m -Xmx6144m -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:NewRatio=2 -XX:+AggressiveOpts -Xrs -XX:PerfDataSamplingInterval=500 -XX:MaxPermSize=512m"
3) Enable Native JAI and Native JAI ImageIO
4) Enable Tile caching into External PostgreSQL database

For some reason, Server is not consuming memory no mater how many clients are requesting maps. There is always more than 16GB free memory (attachment free.jpg).
If you can give as any hint, idea or help, we would be thankful.

Regards,

Ana Boljevic

--



_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users


-- 
Angelos Tzotsos, PhD
Charter Member
Open Source Geospatial Foundation
http://users.ntua.gr/tzotsos

_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users




--
Cell: <a href="tel:%28973%29%20931-0580" value="&#43;19739310580" target="_blank" moz-do-not-send="true"> 973-931-0580


_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users





--
Cell: <a href="tel:%28973%29%20931-0580" value="&#43;19739310580" target="_blank" moz-do-not-send="true"> 973-931-0580





--
Cell: 973-931-0580




_______________________________________________
geonode-users mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geonode-users