sprint results: maven build improvements part 2/2

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

sprint results: maven build improvements part 2/2

jody.garnett
GeoNetwork PSC:

The next proposal requires your input, it contains a proposal to resolve some of the complexity in the web-app pom.xml and should result in an easier/faster build process for everyone.

I would like confirmation on the approach before sinking time into implementation :)

Please check out the proposal here:


_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: sprint results: maven build improvements part 2/2

jody.garnett
Thanks to Juan for a quick review.

Proposal has been updated to stage sql migration scripts into a distinct target/migrate folder (I was unaware that geonetwork requires these on the filesystem).
--
Jody Garnett


On Tue, 14 Jul 2020 at 08:04, Jody Garnett <[hidden email]> wrote:
GeoNetwork PSC:

The next proposal requires your input, it contains a proposal to resolve some of the complexity in the web-app pom.xml and should result in an easier/faster build process for everyone.

I would like confirmation on the approach before sinking time into implementation :)

Please check out the proposal here:


_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: sprint results: maven build improvements part 2/2

Jose Garcia
Hi Jody

Thanks for the proposal. It looks good to me, any improvement on the build is welcome. Some comments:

1) In this workflow web-ui resources, and schema plugins folders would be used directly, no process-resources required.

Note that currently the schematron validation files (sch) files get compiled into xslt in the schema plugins folder that is copied to  web during process-resources in GeoNetwork startup. With this change will be created in the schema sources folder, I guess could be solved adding in .gitignore some rule to exclude xslt files from the schematron folders.

2) Related to the discussion items:

However there is a duplication of effort, and there may be some frustration testing the war bundle as a result.

I think it can be relevant to elaborate this a bit more in the proposal, so it's clear about the duplication of effort.

- Q: The gn database is created in the current directory, can this move to target folder?
A: The only reason to keep in sources can be to avoid loosing the info with mvn clean, but maybe then can be configured a profile to externalize the data directory?

I guess as you pointed out in the proposal, if it can be configured to create it in an external directory using a maven profile, sounds a good option.

Regards,
Jose García

On Tue, Jul 14, 2020 at 5:27 PM Jody Garnett <[hidden email]> wrote:
Thanks to Juan for a quick review.

Proposal has been updated to stage sql migration scripts into a distinct target/migrate folder (I was unaware that geonetwork requires these on the filesystem).
--
Jody Garnett


On Tue, 14 Jul 2020 at 08:04, Jody Garnett <[hidden email]> wrote:
GeoNetwork PSC:

The next proposal requires your input, it contains a proposal to resolve some of the complexity in the web-app pom.xml and should result in an easier/faster build process for everyone.

I would like confirmation on the approach before sinking time into implementation :)

Please check out the proposal here:
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork


--
Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: <a href="tel:+31318416664" style="font-family:Helvetica,Arial,sans-serif" target="_blank">+31 (0)318 416664

Please consider the environment before printing this email.


_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: sprint results: maven build improvements part 2/2

jody.garnett
Thanks Jose that is helpful.

Some comments inline:

1) In this workflow web-ui resources, and schema plugins folders would be used directly, no process-resources required.

Note that currently the schematron validation files (sch) files get compiled into xslt in the schema plugins folder that is copied to  web during process-resources in GeoNetwork startup. With this change will be created in the schema sources folder, I guess could be solved adding in .gitignore some rule to exclude xslt files from the schematron folders.

That is ... odd. I wonder why it is not producing xslt files in this location already (must be an accident or undocument side effect of having so much duplication in the current build). 
I am not sure I can untangle this until we are doing the work, your suggestion of using .gitignore is fine, but I would prefer not to have src folders being modified at runtime if we can avoid it. We would also need to avoid copying these generated files into target/schemas when staging content for war.

Does GeoNetwork only compile these files on startup? Or does it watch for changes ...

2) Related to the discussion items:

However there is a duplication of effort, and there may be some frustration testing the war bundle as a result.

I think it can be relevant to elaborate this a bit more in the proposal, so it's clear about the duplication of effort.

I am trying to say that using jetty:run does not exactly test the same thing using jetty:run-war.

To use your schematron example:

1. If you were using jetty:run and process-resources to try out a rule change
2. And then packaged a war for a customer
3. You would of missed out including your rule change in the war

To fix:

4. Use install to package the plugin jar and zip
5. And then package war for customer (it will pick up the rule change which is now included in the zip)

I hope this example makes it more clear? Two ways of doing things is duplication.

- Q: The gn database is created in the current directory, can this move to target folder?
A: The only reason to keep in sources can be to avoid loosing the info with mvn clean, but maybe then can be configured a profile to externalize the data directory?

I guess as you pointed out in the proposal, if it can be configured to create it in an external directory using a maven profile, sounds a good option.

Yes I proposed a web/data_directory to make this more clear (and provide a single location to .gitignore)


Regards,
Jose García

On Tue, Jul 14, 2020 at 5:27 PM Jody Garnett <[hidden email]> wrote:
Thanks to Juan for a quick review.

Proposal has been updated to stage sql migration scripts into a distinct target/migrate folder (I was unaware that geonetwork requires these on the filesystem).
--
Jody Garnett


On Tue, 14 Jul 2020 at 08:04, Jody Garnett <[hidden email]> wrote:
GeoNetwork PSC:

The next proposal requires your input, it contains a proposal to resolve some of the complexity in the web-app pom.xml and should result in an easier/faster build process for everyone.

I would like confirmation on the approach before sinking time into implementation :)

Please check out the proposal here:
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork


--
Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: <a href="tel:+31318416664" style="font-family:Helvetica,Arial,sans-serif" target="_blank">+31 (0)318 416664

Please consider the environment before printing this email.


_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: sprint results: maven build improvements part 2/2

Jose Garcia
Hi

See inline.

On Mon, Jul 20, 2020 at 8:29 PM Jody Garnett <[hidden email]> wrote:
Thanks Jose that is helpful.

Some comments inline:

1) In this workflow web-ui resources, and schema plugins folders would be used directly, no process-resources required.

Note that currently the schematron validation files (sch) files get compiled into xslt in the schema plugins folder that is copied to  web during process-resources in GeoNetwork startup. With this change will be created in the schema sources folder, I guess could be solved adding in .gitignore some rule to exclude xslt files from the schematron folders.

That is ... odd. I wonder why it is not producing xslt files in this location already (must be an accident or undocument side effect of having so much duplication in the current build). 
I am not sure I can untangle this until we are doing the work, your suggestion of using .gitignore is fine, but I would prefer not to have src folders being modified at runtime if we can avoid it. We would also need to avoid copying these generated files into target/schemas when staging content for war.

Does GeoNetwork only compile these files on startup? Or does it watch for changes ...

Doesn't watch the files for changes, but can trigger a service to re-create them.
 

2) Related to the discussion items:

However there is a duplication of effort, and there may be some frustration testing the war bundle as a result.

I think it can be relevant to elaborate this a bit more in the proposal, so it's clear about the duplication of effort.

I am trying to say that using jetty:run does not exactly test the same thing using jetty:run-war.

To use your schematron example:

1. If you were using jetty:run and process-resources to try out a rule change
2. And then packaged a war for a customer
3. You would of missed out including your rule change in the war

To fix:

4. Use install to package the plugin jar and zip
5. And then package war for customer (it will pick up the rule change which is now included in the zip)

I hope this example makes it more clear? Two ways of doing things is duplication.

- Q: The gn database is created in the current directory, can this move to target folder?
A: The only reason to keep in sources can be to avoid loosing the info with mvn clean, but maybe then can be configured a profile to externalize the data directory?

I guess as you pointed out in the proposal, if it can be configured to create it in an external directory using a maven profile, sounds a good option.

Yes I proposed a web/data_directory to make this more clear (and provide a single location to .gitignore)


Regards,
Jose García

On Tue, Jul 14, 2020 at 5:27 PM Jody Garnett <[hidden email]> wrote:
Thanks to Juan for a quick review.

Proposal has been updated to stage sql migration scripts into a distinct target/migrate folder (I was unaware that geonetwork requires these on the filesystem).
--
Jody Garnett


On Tue, 14 Jul 2020 at 08:04, Jody Garnett <[hidden email]> wrote:
GeoNetwork PSC:

The next proposal requires your input, it contains a proposal to resolve some of the complexity in the web-app pom.xml and should result in an easier/faster build process for everyone.

I would like confirmation on the approach before sinking time into implementation :)

Please check out the proposal here:
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork


--
Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: <a href="tel:+31318416664" style="font-family:Helvetica,Arial,sans-serif" target="_blank">+31 (0)318 416664

Please consider the environment before printing this email.


--
Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: <a href="tel:+31318416664" style="font-family:Helvetica,Arial,sans-serif" target="_blank">+31 (0)318 416664

Please consider the environment before printing this email.


_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork