[OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+-------------------
 Reporter:  jive           |      Owner:  sac@…
     Type:  task           |     Status:  new
 Priority:  normal         |  Milestone:
Component:  Systems Admin  |   Keywords:
---------------------------+-------------------
 Planet has informed us that they would like to retire
 repo.boundlessgeo.com (an artifactory instance) used by the GeoServer
 project. The GeoServer PSC would like to ask OSGeo if it is possible to
 host an artifact manager on OSGeo hardware?

 As discussed in 2019 (see https://trac.osgeo.org/osgeo/ticket/2261) this
 is a good oppertunity to setup an  artifact manager to replace
 http://download.osgeo.org/webdav/geotools/

 There are two good options:

 * https://jfrog.com/open-source/ - easy to manage
 * https://www.sonatype.com/download-oss-sonatype - does slightly more

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by jive):

 This is a somewhat time sensitive activity as we would like to rescue the
 contents of repo.boundlessgeo.com (mostly prior release of GeoServer and
 GeoWebCache).

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:1>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by jive):

 As noted in email discussion these repository managers can be configured
 to manage maven, docker, node, etc... and may be of interest to a sider
 group of osgeo projects.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:2>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by wildintellect):

 Possible: Yes
 How: we need some members to step up to set up a container dedicated to
 this purpose and hopefully have some expertise or at least time to vet the
 options.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:3>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by robe):

 jive -- as a clean replacement sounds like might be better to go with
 jfrog since that is what repo.boundlessgeo.com is using.

 What does sonatype do additional that jfrog doesn't that would be a
 possible need?

 I'd be willing to take a stab at installing it -- you have details of how
 we can pull over the artifacts from boundless?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:4>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by jive):

 Answering your question *What does sonatype do additional that jfrog
 doesn't that would be a possible need?*

 Frog artifactory:

 * https://jfrog.com/open-source/
 * personal experience: easier administration screens to manage
 * Only supports java projects (Maven, Gradle and Ivy)
 * Used by repo.boundlessgeo.com (after a poor experience with cloud
 hosting)

 Nexus:

 * https://www.sonatype.com/download-oss-sonatype
 * personal experience: more complicated administration screens
 * This is the clear choice for OSGeo as it offers capabilities for more
 than just java projects (docker images, node packages, ruby, python, go,
 etc...)
 * Used by eclipse foundation (with their strong emphasis on control),

 I have personally used both as an administrator (not installing but
 managing).

 Links:

 * https://www.sonatype.com/nexus-repository-oss
 * https://sonatype.drift.click/oss-vs-pro
 * https://info.sonatype.com/nexus-vs-artifactory

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:5>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by jive):

 I think to copy the contents Tom Ingold did:

 1. Set up repo as a cache of old repo
 2. Have a script that accessed all the content via the new URL
 3. Turn of the "cache" and keep the content

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:6>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by jive):

 So what happens next?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:7>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+--------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+--------------------

Comment (by robe):

 jive,

 I'm planning to setup osgeo3 this week and thinking I would set up a
 container for nexus and jfrog.

 I'm thinking maybe it would be easier if I setup jfrog first that way you
 have a clean migration from boundless?  and then we can do nexus and
 slowly port over?

 I don't think it's necessary for all projects to use the same thing so if
 you feel more comfortable with artifactory I'd say you should use that.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:8>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------
Changes (by robe):

 * milestone:   => Sysadmin Contract 2020-I


--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:9>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by robe):

 Jody - I setup nexus using the docker container from here
 https://hub.docker.com/r/sonatype/nexus3/ - I sent you the login details

 https://nexus.osgeo.org

 It's running in a dedicated LXD OS container so we could put other dockery
 stuff in there you need.

 It's running on the new osgeo3 lxd host I set up today.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:10>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by robe):

 nexus supports LDAP so I configured it to use OSGeo LDAP and set jive,
 wildintellect, strk as an admins.

 There is a setting in the LDAP config to use LDAP groups for role
 management.  But I wasn't sure if we wanted to do that for the first go
 round.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:11>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by robe):

 based on popular demand I have added

 https://repo.osgeo.org  as an alias for https://nexus.osgeo.org

 we can get rid of nexus but that's what the container is called and the
 config I have at moment and some other names were proposed like
 docker.osgeo.org, apt.osgeo.org etc.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:12>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by jive):

 Thanks for the hard work, I am collecting notes on how to transfer
 content.

 1. webdav

 This [https://community.sonatype.com/t/nexus-repository-artifact-import-
 scripts/1254 script] would like to be run on osgeo machine hosting webdav
 folder.

 This replies on direct file system access and should be fairly quick.

 * from: http://download.osgeo.org/webdav/geotools
 * to: https://repo.osgeo.org/geotools-releases

 Once content is migrated we can redirect
 http://download.osgeo.org/webdav/geotools/ --> repo.osgeo.org/release


 2. repo.boundlessgeo.com to

 This [https://github.com/simpligility/maven-repository-tools/tree/master
 /maven-repository-provisioner maven repository provisioner] can be run
 from anywhere, and just uses maven API.

 * from: https://repo.boundlessgeo.com/artifactory/release
 * to: https://repo.osgeo.org/geoserver-releases

 I expect this will be quite bandwidth intensive, as such I would prefer to
 run this script on osgeo hardware.

 3. when complete

 Update the builds for osgeo projects and communicate the change to
 downstream applications

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:13>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by robe):

 jive are you all set here?  Last we discussed you had root access to nexus
 container and download container and can install anything you need on the
 nexus container.

 I also want to do an upgrade of nexus install to deal with vulnerability
 that was announced a couple of days ago.  I'm not sure you have any groovy
 scripts so may be a non-issue.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:14>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by robe):

 I did the upgrade on nexus.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:15>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by jive):

 Tom Ingold was able to transfer content from
 ​https://repo.boundlessgeo.com/artifactory/release to Geoserver-releases
 (no idea why that ended up with a capital but it does not matter so much
 as it mostly an internal detail).  The maven repository provisioner did
 not work, but he was able to figure some scripts out (wget, find, xargs
 and curl).

 Initial testing found that some things were missed (apache beanutils) so
 some testing is needed.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:16>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by jive):

 I have written up a wiki page https://wiki.osgeo.org/wiki/SAC:Repo
 describing how projects can join the repository.

 We have our initial builds working from GeoTools, I think this issue can
 be closed!

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:17>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  new
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:
 Keywords:                 |
---------------------------+---------------------------------------

Comment (by jive):

 aside: I was able to complete migrating contents from webdav folder.

 Should we make plans to shutoff that webdav folder, or redirect to
 https://repo.osgeo.org/repository/release/ ?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:18>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
Reply | Threaded
Open this post in threaded view
|

Re: [OSGeo] #2410: artifact manager to replace download.osgeo.org/webdav/geotools

OSGeo
In reply to this post by OSGeo
#2410: artifact manager to replace download.osgeo.org/webdav/geotools
---------------------------+---------------------------------------
 Reporter:  jive           |       Owner:  sac@…
     Type:  task           |      Status:  closed
 Priority:  normal         |   Milestone:  Sysadmin Contract 2020-I
Component:  Systems Admin  |  Resolution:  fixed
 Keywords:                 |
---------------------------+---------------------------------------
Changes (by robe):

 * status:  new => closed
 * resolution:   => fixed


--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2410#comment:19>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.

_______________________________________________
Sac mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/sac
12