A large number of NetCDF non-release problems

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

A large number of NetCDF non-release problems

niuben1055
gs-main-2.15.0 --> ResourcePool --> getGridCoverageReader
hintCoverageReaderCache --> CoverageHintReaderCache

SoftReference  
  Soft Connection Not Released
  Release when memory is full
  Not full memory has exceeded system limits

How to solve this problem?





--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: A large number of NetCDF non-release problems

Daniele Romagnoli-3
Hi,
could you please provide some more background info and details on what are you doing and when the problem occurs?
Which type of store and how many of them are you using? (imagemosaic vs netcdf store)
Are you using any specific datastore configuration for them?
When the problem occurs and how to replicate it?

Please, let us know.
Best Regards,
Daniele




On Wed, Apr 10, 2019 at 3:02 PM niuben1055 <[hidden email]> wrote:
gs-main-2.15.0 --> ResourcePool --> getGridCoverageReader
hintCoverageReaderCache --> CoverageHintReaderCache

SoftReference 
  Soft Connection Not Released
  Release when memory is full
  Not full memory has exceeded system limits

How to solve this problem?





--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: A large number of NetCDF non-release problems

niuben1055

Thanks a lot for your response.

I used RedHat Linux, installed geoserver 2.13.0 with Tomcat 8.5.34, and
installed NetCDF library.
I used REST to add NetCDF data files to geoserver. When the files are over
1000, geoserver dumped. I found some messages in the log file:

27-Feb-2019 16:50:05.674 error [http-nio-8080-Acceptor-0]
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:482)
        at java.lang.Thread.run(Thread.java:748)

I checked the geoserver's source code and found it uses java SoftRederence
to release resource, but the memories the connections used are very small,
and it cannot make JVM automatically release memory. So when I add too many
NetCDF files to geoserver, its opened file number would be more than the
open file limits of Linux, thus the geoserver could not be used any more.

And then I used postgis database and put mddatastore.properties in the
directory of NetCDF file. I found connection could not be created when the
connection nubers are over 95. Then I checked the postgis connection status.
It had reached the max connection number.

I need to add over 1,000,000 NetCDF files to geoserver. My question is how
to accomplish that.

I'm expectiong your response.



--
Sent from: http://osgeo-org.1560.x6.nabble.com/GeoServer-User-f3786390.html


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users