Harvesting from a web-accessible folder in an S3 bucket

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Harvesting from a web-accessible folder in an S3 bucket

Jo Cook
Hi,

I'm wondering if anyone has had any luck harvesting metadata from a
web-accessible folder stored in an S3 bucket. I'm using Geonetwork 3.4 for
testing, and I'm not getting any records returned, despite the fact that
there are xml files in the bucket. I've looked at permissions, and
autogenerating an index.html, but I think S3 buckets are different to
normal web folders, and I'm wondering if it's even possible to harvest from
them.

The bucket I am trying to harvest from is
https://s3-eu-west-1.amazonaws.com/mmo-defra-catalogue/. There's a
generated index.html at
https://s3-eu-west-1.amazonaws.com/mmo-defra-catalogue/index.html.

If it's not possible, is it something I could submit as a potential
enhancement?

Thanks

Jo



--
*Jo Cook*
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent
responses at that time, please visit support.astuntechnology.com or phone
our office on 01372 744009

--
--
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey,
KT18 7RL, UK 
t:+44 1372 744 009 w: astuntechnology.com
<http://astuntechnology.com/> twitter:@astuntech
<https://twitter.com/astuntech>



iShare - enterprise geographic
intelligence platform <https://astuntechnology.com/ishare/>
GeoServer,
PostGIS and QGIS training <https://astuntechnology.com/services/#training>

Helpdesk and customer portal
<http://support.astuntechnology.com/support/login>




Company registration
no. 5410695. Registered in England and Wales. Registered office: 120 Manor
Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: Harvesting from a web-accessible folder in an S3 bucket

Jose Garcia
Hi Jo

I have try it with these results:

- Type of protocol: WAF -> 0 results (no error in the log)

- Type of protocol: WebDav -> Error:

Host name 's3-eu-west-1.amazonaws.com' does not match the certificate
subject provided by the peer (CN=*.s3-eu-west-1.amazonaws.com, O=Amazon.com
Inc., L=Seattle, ST=Washington, C=US)

I need to debug this a bit to get clear why is not getting any record.

Regards,
Jose García

On Mon, Apr 9, 2018 at 11:07 AM, Jo Cook <[hidden email]> wrote:

> Hi,
>
> I'm wondering if anyone has had any luck harvesting metadata from a
> web-accessible folder stored in an S3 bucket. I'm using Geonetwork 3.4 for
> testing, and I'm not getting any records returned, despite the fact that
> there are xml files in the bucket. I've looked at permissions, and
> autogenerating an index.html, but I think S3 buckets are different to
> normal web folders, and I'm wondering if it's even possible to harvest from
> them.
>
> The bucket I am trying to harvest from is
> https://s3-eu-west-1.amazonaws.com/mmo-defra-catalogue/. There's a
> generated index.html at
> https://s3-eu-west-1.amazonaws.com/mmo-defra-catalogue/index.html.
>
> If it's not possible, is it something I could submit as a potential
> enhancement?
>
> Thanks
>
> Jo
>
>
>
> --
> *Jo Cook*
> t:+44 7930 524 155/twitter:@archaeogeek
> Please note that currently I do not work on Friday afternoons. For urgent
> responses at that time, please visit support.astuntechnology.com or phone
> our office on 01372 744009
>
> --
> --
> Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey,
> KT18 7RL, UK
> t:+44 1372 744 009 w: astuntechnology.com
> <http://astuntechnology.com/> twitter:@astuntech
> <https://twitter.com/astuntech>
>
>
>
> iShare - enterprise geographic
> intelligence platform <https://astuntechnology.com/ishare/>
> GeoServer,
> PostGIS and QGIS training <https://astuntechnology.com/services/#training>
>
> Helpdesk and customer portal
> <http://support.astuntechnology.com/support/login>
>
>
>
>
> Company registration
> no. 5410695. Registered in England and Wales. Registered office: 120 Manor
> Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> GeoNetwork-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> GeoNetwork OpenSource is maintained at http://sourceforge.net/
> projects/geonetwork
>



--













*Vriendelijke groeten / Kind regards,Jose García
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664> <https://www.facebook.com/geocatbv>
<https://twitter.com/geocat_bv>
<https://plus.google.com/u/1/+GeocatNetbv/posts>Please consider the
environment before printing this email.*
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork