Imagemosaic and TimestampFileNameExtractorSPI indexing

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Imagemosaic and TimestampFileNameExtractorSPI indexing

alexander-petkov
Is it possible to use subdirectory/filename in order to extract a
timestamp during ImageMosaic coverage configuration?

Or, is only the File name used for timestamp extraction?

I have sub directories and files containing the following naming format:

rtma2p5.20190731/rtma2p5.t00z.2dvaranl_ndfd.grb2_wexp
where t00 is from 00 to 23.

Thanks in advance!!

Alex


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Imagemosaic and TimestampFileNameExtractorSPI indexing

Daniele Romagnoli-3
Hi Alexander,
you may combine custom format and fullPath parameter.
There is an example in the GeoTools TimestampFileNameExtractorTest:

I'm not very familiar with regex so I cannot help you too much with that part. I always have to check the rules, each time I need a complex regex, which doesn't occur very often.
Once you have identified your regex, you can use the propertiesCollector with this String for your file rtma2p5.20190731/rtma2p5.t00z.2dvaranl_ndfd.grb2_wexp:

"regex=yourmatchingregex,format=yyyyMMdd't'HH'z',fullPath=true"

Please, let us know.
Regards,
Daniele


 

On Tue, Aug 13, 2019 at 7:58 PM Alexander Petkov <[hidden email]> wrote:
Is it possible to use subdirectory/filename in order to extract a
timestamp during ImageMosaic coverage configuration?

Or, is only the File name used for timestamp extraction?

I have sub directories and files containing the following naming format:

rtma2p5.20190731/rtma2p5.t00z.2dvaranl_ndfd.grb2_wexp
where t00 is from 00 to 23.

Thanks in advance!!

Alex


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Imagemosaic and TimestampFileNameExtractorSPI indexing

alexander-petkov
Thank you Daniele, for your help!! The test example you provided was
very helpful to get me started.

This entry in timeregex.properties works:

regex=([0-9]{8})(?:/rtma2p5.)(t[0-9]{2}z),format=yyyyMMdd't'HH'z',fullPath=true

Mosaic granules  are in subdirectories as follows:

rtma2p5.20190731/rtma2p5.t00z.2dvaranl_ndfd.grb2_wexp

where time varies from 00 to 23 (for every hour)

It took a while to write a regex pattern. It took some time studying
regex rules.

I also tested the pattern on https://www.compilejava.net with this test:

===================
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.regex.PatternSyntaxException;

public class TestPatternExtraction
{
  public static void main(String[] args)
  {
    String expr="([0-9]{8})(?:/rtma2p5.)(t[0-9]{2}z)";
    //String expr="(.*[0-9]{8}.*t[0-9]{2}z.*)";
    String name=
"/wfas/data/rtma/rhm/rtma2p5.20190731/rtma2p5.t03z.2dvaranl_ndfd.grb2_wexp";
    Pattern regex = Pattern.compile(expr);
    Matcher regexMatcher = regex.matcher(name);
    while (regexMatcher.find()) {
      try{
        for (int i = 1; i <= regexMatcher.groupCount(); i++) {
            System.out.println("matched text: " + regexMatcher.group(i));
            System.out.println("match start: " + regexMatcher.start(i));
            System.out.println("match end: " + regexMatcher.end(i));
        }
      }  catch (PatternSyntaxException ex) {
         // Syntax error in the regular expression
      }//end try
   }//end while
}//end main
}
=======================

This list conversation was also helpful:
https://sourceforge.net/p/geoserver/mailman/message/36061535/

As well as firing up Eclipse and tracing through the code.

Kind regards,
Alex


On 8/14/19, Daniele Romagnoli <[hidden email]> wrote:

> Hi Alexander,
> you may combine custom *format *and *fullPath* parameter.
> There is an example in the GeoTools TimestampFileNameExtractorTest:
> https://github.com/geotools/geotools/blob/21.2/modules/plugin/imagemosaic/src/test/java/org/geotools/gce/imagemosaic/properties/time/TimestampFileNameExtractorTest.java#L110
>
>
> I'm not very familiar with regex so I cannot help you too much with that
> part. I always have to check the rules, each time I need a complex regex,
> which doesn't occur very often.
> Once you have identified your regex, you can use the propertiesCollector
> with this String for your file
> *rtma2p5.20190731/rtma2p5.t00z.2dvaranl_ndfd.grb2_wexp*:
>
> "regex=yourmatchingregex,format=yyyyMMdd't'HH'z',fullPath=true"
>
> Please, let us know.
> Regards,
> Daniele


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users