Updating imagemosaic index after granule updating

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Updating imagemosaic index after granule updating

Micho García
Hi,

I'm Micho, I work at geomati.co, I am an OSGeo charter member and first of all my total admiration for your work developing and maintaining GeoServer.

We are using GeoServer maybe in a weird use case, we are publishing geotiff as granules of an imagemosaic with the time dimension. When we publish the first granule it creates the store and next granules go adding to this store. The problem that we have found is because we usually update the same granule several times but the index doesn't refresh. We found that in the store folder in the data_dir is the correct last geotiff but geoserver is publishing other geotiff. We have checked when we force the geoserver reloads geoserver publishes the correct geotiff. 

It looks like that the index isn't refreshing correctly, maybe because we are updating always the same granule. 

We are using gsconfig-py3 to every REST operation to GeoServer. In fact, the method which publishes the granule is the `catalog.harvest_uploadgranule`.


I was looking for some info, and I've found some comments in any other forums, now I'm trying to add the recalculate parameter to see it this parameter forces the index recalculation.


Would I like if someone has experienced this issue before? Is there any way to force the update of the imagemosaic index?.

A lot of thanks.

Best.

Micho Garcia


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Updating imagemosaic index after granule updating

Daniele Romagnoli-3
Hi Micho,
It looks like some other people experienced the same issue:

Did you already try the same approach?
Please, let us know.

Cheers,
Daniele

On Thu, Apr 4, 2019 at 12:06 PM Micho García <[hidden email]> wrote:
Hi,

I'm Micho, I work at geomati.co, I am an OSGeo charter member and first of all my total admiration for your work developing and maintaining GeoServer.

We are using GeoServer maybe in a weird use case, we are publishing geotiff as granules of an imagemosaic with the time dimension. When we publish the first granule it creates the store and next granules go adding to this store. The problem that we have found is because we usually update the same granule several times but the index doesn't refresh. We found that in the store folder in the data_dir is the correct last geotiff but geoserver is publishing other geotiff. We have checked when we force the geoserver reloads geoserver publishes the correct geotiff. 

It looks like that the index isn't refreshing correctly, maybe because we are updating always the same granule. 

We are using gsconfig-py3 to every REST operation to GeoServer. In fact, the method which publishes the granule is the `catalog.harvest_uploadgranule`.


I was looking for some info, and I've found some comments in any other forums, now I'm trying to add the recalculate parameter to see it this parameter forces the index recalculation.


Would I like if someone has experienced this issue before? Is there any way to force the update of the imagemosaic index?.

A lot of thanks.

Best.

Micho Garcia
_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Updating imagemosaic index after granule updating

Simone Giannecchini
Minor observation,
I would not use a shapefile as mutable index, use Postgis instead.

If by adding granules the BBOX changes you will have to recalculate.
As an alternative you can set the bbox initially in GeoServer to a
value that is big enough as you will not need to recalculate
afterwards.

Regards,
Simone Giannecchini
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati personali
(Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati
“GDPR”), si precisa che ogni circostanza inerente alla presente email
(il suo contenuto, gli eventuali allegati, etc.) è un dato la cui
conoscenza è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a
cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato
se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential
or otherwise protected from disclosure. We remind that - as provided
by European Regulation 2016/679 “GDPR” - copying, dissemination or use
of this e-mail or the information herein by anyone other than the
intended recipient is prohibited. If you have received this email by
mistake, please notify us immediately by telephone or e-mail.

On Fri, Apr 5, 2019 at 4:36 PM Daniele Romagnoli
<[hidden email]> wrote:

>
> Hi Micho,
> It looks like some other people experienced the same issue:
> https://gis.stackexchange.com/questions/207460/geoserver-imagemosaic-index-is-not-updating-with-curl
>
> Did you already try the same approach?
> Please, let us know.
>
> Cheers,
> Daniele
>
> On Thu, Apr 4, 2019 at 12:06 PM Micho García <[hidden email]> wrote:
>>
>> Hi,
>>
>> I'm Micho, I work at geomati.co, I am an OSGeo charter member and first of all my total admiration for your work developing and maintaining GeoServer.
>>
>> We are using GeoServer maybe in a weird use case, we are publishing geotiff as granules of an imagemosaic with the time dimension. When we publish the first granule it creates the store and next granules go adding to this store. The problem that we have found is because we usually update the same granule several times but the index doesn't refresh. We found that in the store folder in the data_dir is the correct last geotiff but geoserver is publishing other geotiff. We have checked when we force the geoserver reloads geoserver publishes the correct geotiff.
>>
>> It looks like that the index isn't refreshing correctly, maybe because we are updating always the same granule.
>>
>> We are using gsconfig-py3 to every REST operation to GeoServer. In fact, the method which publishes the granule is the `catalog.harvest_uploadgranule`.
>>
>> https://github.com/dimitri-justeau/gsconfig-py3/blob/master/src/geoserver/catalog.py#L554
>>
>> I was looking for some info, and I've found some comments in any other forums, now I'm trying to add the recalculate parameter to see it this parameter forces the index recalculation.
>>
>> https://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html#recalculate
>>
>> Would I like if someone has experienced this issue before? Is there any way to force the update of the imagemosaic index?.
>>
>> A lot of thanks.
>>
>> Best.
>>
>> Micho Garcia
>> http://geomati.co
>> _______________________________________________
>> Geoserver-users mailing list
>>
>> Please make sure you read the following two resources before posting to this list:
>> - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
>> - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html
>>
>> If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
>>
>>
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
> --
> Regards,
> Daniele Romagnoli
> ==
> GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
> ==
>
> Ing. Daniele Romagnoli
> Senior Software Engineer
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax:      +39 0584 1660272
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> -------------------------------------------------------
>
> Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>
> This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> _______________________________________________
> Geoserver-users mailing list
>
> Please make sure you read the following two resources before posting to this list:
> - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
> - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html
>
> If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
>
>
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geoserver-users


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Updating imagemosaic index after granule updating

Micho García
Hi Simone,

thanks!!, changing the index to a PosGIS database it has been the only way to solve the problem. With the shapefile, including using the recalculate parameter it doesn't work ever.

Best
Micho Garcia

El mar., 9 abr. 2019 a las 17:20, Simone Giannecchini (<[hidden email]>) escribió:
Minor observation,
I would not use a shapefile as mutable index, use Postgis instead.

If by adding granules the BBOX changes you will have to recalculate.
As an alternative you can set the bbox initially in GeoServer to a
value that is big enough as you will not need to recalculate
afterwards.

Regards,
Simone Giannecchini
==
GeoServer Professional Services from the experts!
Visit http://bit.ly/gs-services for more information.
==
Ing. Simone Giannecchini
@simogeo
Founder/Director

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39  333 8128928

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------
Con riferimento alla normativa sul trattamento dei dati personali
(Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati
“GDPR”), si precisa che ogni circostanza inerente alla presente email
(il suo contenuto, gli eventuali allegati, etc.) è un dato la cui
conoscenza è riservata al/i solo/i destinatario/i indicati dallo
scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a
cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato
se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential
or otherwise protected from disclosure. We remind that - as provided
by European Regulation 2016/679 “GDPR” - copying, dissemination or use
of this e-mail or the information herein by anyone other than the
intended recipient is prohibited. If you have received this email by
mistake, please notify us immediately by telephone or e-mail.

On Fri, Apr 5, 2019 at 4:36 PM Daniele Romagnoli
<[hidden email]> wrote:
>
> Hi Micho,
> It looks like some other people experienced the same issue:
> https://gis.stackexchange.com/questions/207460/geoserver-imagemosaic-index-is-not-updating-with-curl
>
> Did you already try the same approach?
> Please, let us know.
>
> Cheers,
> Daniele
>
> On Thu, Apr 4, 2019 at 12:06 PM Micho García <[hidden email]> wrote:
>>
>> Hi,
>>
>> I'm Micho, I work at geomati.co, I am an OSGeo charter member and first of all my total admiration for your work developing and maintaining GeoServer.
>>
>> We are using GeoServer maybe in a weird use case, we are publishing geotiff as granules of an imagemosaic with the time dimension. When we publish the first granule it creates the store and next granules go adding to this store. The problem that we have found is because we usually update the same granule several times but the index doesn't refresh. We found that in the store folder in the data_dir is the correct last geotiff but geoserver is publishing other geotiff. We have checked when we force the geoserver reloads geoserver publishes the correct geotiff.
>>
>> It looks like that the index isn't refreshing correctly, maybe because we are updating always the same granule.
>>
>> We are using gsconfig-py3 to every REST operation to GeoServer. In fact, the method which publishes the granule is the `catalog.harvest_uploadgranule`.
>>
>> https://github.com/dimitri-justeau/gsconfig-py3/blob/master/src/geoserver/catalog.py#L554
>>
>> I was looking for some info, and I've found some comments in any other forums, now I'm trying to add the recalculate parameter to see it this parameter forces the index recalculation.
>>
>> https://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html#recalculate
>>
>> Would I like if someone has experienced this issue before? Is there any way to force the update of the imagemosaic index?.
>>
>> A lot of thanks.
>>
>> Best.
>>
>> Micho Garcia
>> http://geomati.co
>> _______________________________________________
>> Geoserver-users mailing list
>>
>> Please make sure you read the following two resources before posting to this list:
>> - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
>> - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html
>>
>> If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
>>
>>
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
> --
> Regards,
> Daniele Romagnoli
> ==
> GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
> ==
>
> Ing. Daniele Romagnoli
> Senior Software Engineer
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax:      +39 0584 1660272
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> -------------------------------------------------------
>
> Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>
> This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> _______________________________________________
> Geoserver-users mailing list
>
> Please make sure you read the following two resources before posting to this list:
> - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
> - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html
>
> If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
>
>
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geoserver-users


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users