[QGIS-Developer] Delaying 3.10.1?

classic Classic list List threaded Threaded
58 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[QGIS-Developer] Delaying 3.10.1?

Nyall Dawson
Hi list,

Just wondering if there's any chance of a small delay in tagging 3.10.1?

I'm trying to get to the bottom of this (serious) issue in the Proj 6
based builds (which will be 3.10.1 for Windows users):
https://github.com/qgis/QGIS/issues/30569

It's very serious, and in my view should be a release blocker.

It's possibly an underlying issue in proj, or in how QGIS is
(mis?)using the proj api:

https://github.com/OSGeo/PROJ/issues/1736

But I'd love a couple of days to keep digging, if that's at all possible....

Nyall
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Andreas Neumann-4

Hi Nyall,

This sounds to me like a good reason to delay the dot release. But ultimately, Jürgen should decide this.

Thanks for working on it!

Andreas

On 2019-11-22 08:17, Nyall Dawson wrote:

Hi list,

Just wondering if there's any chance of a small delay in tagging 3.10.1?

I'm trying to get to the bottom of this (serious) issue in the Proj 6
based builds (which will be 3.10.1 for Windows users):
https://github.com/qgis/QGIS/issues/30569

It's very serious, and in my view should be a release blocker.

It's possibly an underlying issue in proj, or in how QGIS is
(mis?)using the proj api:

https://github.com/OSGeo/PROJ/issues/1736

But I'd love a couple of days to keep digging, if that's at all possible....

Nyall
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer



_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Jürgen E. Fischer
In reply to this post by Nyall Dawson
Hi Nyall,

On Fri, 22. Nov 2019 at 17:17:07 +1000, Nyall Dawson wrote:
> But I'd love a couple of days to keep digging, if that's at all possible....

Black friday?  Ok.


Jürgen

--
Jürgen E. Fischer           norBIT GmbH             Tel. +49-4931-918175-31
Dipl.-Inf. (FH)             Rheinstraße 13          Fax. +49-4931-918175-50
Software Engineer           D-26506 Norden            https://www.norbit.de
QGIS release manager (PSC)  Germany                    IRC: jef on FreeNode

norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Juergen Fischer, Nils Kutscher HR: Amtsgericht Aurich HRB 100827
Datenschutzerklaerung: https://www.norbit.de/83/

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

signature.asc (844 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Even Rouault-2
On vendredi 22 novembre 2019 08:43:23 CET Jürgen E. Fischer wrote:
> Hi Nyall,
>
> On Fri, 22. Nov 2019 at 17:17:07 +1000, Nyall Dawson wrote:
> > But I'd love a couple of days to keep digging, if that's at all
> > possible....
> Black friday?  Ok.

Trying to reproduce on my side in isolated way with use of only PROJ-only API

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Andrea Giudiceandrea
In reply to this post by Nyall Dawson
Nyall Dawson wrote
> I'm trying to get to the bottom of this (serious) issue in the Proj 6
> based builds (which will be 3.10.1 for Windows users):
> https://github.com/qgis/QGIS/issues/30569

It seems there are other related bug reports:
https://github.com/qgis/QGIS/issues/32795
https://github.com/qgis/QGIS/issues/32919
https://github.com/qgis/QGIS/issues/32973

Best regards.

Andrea Giudiceandrea




--
Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Borys Jurgiel-4
Excuse me...

Do I see correctly there was a switch from Proj5 to Proj6 between 3.10.0-1 and
3.10.0-2 ?

Two weeks before 3.10.0 we entered that painful hard freeze so we couldn't
revert known regressions (except those introduced after the feature freeze)
and had to release a buggy 3.10.0. Later a few other regressions were found
(including at least one severe: the broken PostGIS in the Browser), so I
assumed 3.10.1 would be the first "stable" 3.10.x (at least free from known
regressions). And now I see a very risky change was introduced between two
builds of the same release?

Please correct me if I'm wrong, as I'd like to avoid a wrongful
overinterpretation.

Borys


Dnia piątek, 22 listopada 2019 13:47:19 CET andreaerdna pisze:

> Nyall Dawson wrote
>
> > I'm trying to get to the bottom of this (serious) issue in the Proj 6
> > based builds (which will be 3.10.1 for Windows users):
> > https://github.com/qgis/QGIS/issues/30569
>
> It seems there are other related bug reports:
> https://github.com/qgis/QGIS/issues/32795
> https://github.com/qgis/QGIS/issues/32919
> https://github.com/qgis/QGIS/issues/32973
>
> Best regards.
>
> Andrea Giudiceandrea
>
>
>
>
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer




_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Saber Razmjooei
Borys,

It is even scarier than this :D

QGIS LTR in Windows has also switched from proj 5.x and gdal 2.x to proj 6.x and gdal 3.x which has resulted in some new bugs.

It will be additional work for package managers and more resources, but it will be good to have some guidelines on LTR and PRs dependencies for packaging.

Regards
Saber


On Fri, 22 Nov 2019 at 17:05, Borys Jurgiel <[hidden email]> wrote:
Excuse me...

Do I see correctly there was a switch from Proj5 to Proj6 between 3.10.0-1 and
3.10.0-2 ?

Two weeks before 3.10.0 we entered that painful hard freeze so we couldn't
revert known regressions (except those introduced after the feature freeze)
and had to release a buggy 3.10.0. Later a few other regressions were found
(including at least one severe: the broken PostGIS in the Browser), so I
assumed 3.10.1 would be the first "stable" 3.10.x (at least free from known
regressions). And now I see a very risky change was introduced between two
builds of the same release?

Please correct me if I'm wrong, as I'd like to avoid a wrongful
overinterpretation.

Borys


Dnia piątek, 22 listopada 2019 13:47:19 CET andreaerdna pisze:
> Nyall Dawson wrote
>
> > I'm trying to get to the bottom of this (serious) issue in the Proj 6
> > based builds (which will be 3.10.1 for Windows users):
> > https://github.com/qgis/QGIS/issues/30569
>
> It seems there are other related bug reports:
> https://github.com/qgis/QGIS/issues/32795
> https://github.com/qgis/QGIS/issues/32919
> https://github.com/qgis/QGIS/issues/32973
>
> Best regards.
>
> Andrea Giudiceandrea
>
>
>
>
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-f4099106.html
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer




_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer


--

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: Delaying 3.10.1?

Nyall Dawson
In reply to this post by Jürgen E. Fischer
On Fri, 22 Nov 2019 at 17:43, Jürgen E. Fischer <[hidden email]> wrote:
>
> Hi Nyall,
>
> On Fri, 22. Nov 2019 at 17:17:07 +1000, Nyall Dawson wrote:
> > But I'd love a couple of days to keep digging, if that's at all possible....
>
> Black friday?  Ok.
>

Thanks for the understanding and flexibility! I'll dig back into this
first thing Monday morning, when I'm back in the office.

(<rant>I briefly tried to setup a proj 6 build on my laptop to test
some things today, but then trying to get a working build in this
setup just made me want to punch something as hard as I could. Can we
delete spatialite yet and move on?</rant>)



>
> Jürgen
>
> --
> Jürgen E. Fischer           norBIT GmbH             Tel. +49-4931-918175-31
> Dipl.-Inf. (FH)             Rheinstraße 13          Fax. +49-4931-918175-50
> Software Engineer           D-26506 Norden            https://www.norbit.de
> QGIS release manager (PSC)  Germany                    IRC: jef on FreeNode
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

[QGIS-Developer] LTR management [was Re: Delaying 3.10.1?]

Even Rouault-2
In reply to this post by Saber Razmjooei
> QGIS LTR in Windows has also switched from proj 5.x and gdal 2.x to
> proj 6.x and gdal 3.x which has resulted in some new bugs.

I see 2 different situations:

- 3.10 was intended to work with GDAL 3 & PROJ 6 (work started with 3.8 if I
remember). OSGeo4W received a late upgrade to them because GRASS 7.8 wasn't
ready yet for them (no offense to the GRASS team !), but devs have been able
to work against GDAL 3 & PROJ 6, so that wasn't unknown territory. Probably
that 3.10.0 could have been delayed while OSGeo4W hadn't switched to GDAL 3/
PROJ 6, but it wasn't clear when GRASS would be out. So overall how the
situation was handled doesn't seem that bad for a release exercising new major
dependencies.

- I do agree that the switch to GDAL 3 & PROJ 6 for a near end-of-life 3.4 LTR
was unfortunate (but more than the timing, that does not seem appropriate for
a release which was not designed originally to work with them) and I felt -0
on that.

> It will be additional work for package managers and more resources, but it
> will be good to have some guidelines on LTR and PRs dependencies for
> packaging.

Each LTR should ideally have its own set of dependencies, allowing controlled
changes of them when needed (adopting a new bugfix version might be OK), and
allowing the current version to use more aggressively new versions of the
dependencies. That said, I'm not a packager, and don't know the effort & cost
associated, but that's clearly an area where the project could/should invest.
Similar situation is likely to happen again in the future.

I'm also wondering if there shouldn't be a vote, from developers, PSC, bug
triagers, testers or probably a mix of them forming a representative body, to
adjudicate:
- dependency upgrades
- decision of releasing a new version (most other OSGeo projects I'm familiar
with rely on a PSC formal vote to issue releases (*)).

Or one might consider a mixed approach to have a good compromise of agility vs
tighter control:
- use time-based approach, as done currently, for non-LTR versions.
- formally approve the release of LTR versions, and important engineering
decisions that affect them, as there are the ones with the most user exposure.
Sometime ago I also suggested to possibly consider 2 phases in a LTR life-
cycle: first half where quite "aggressive" backporting is accepted (if it
doesn't break API, etc..), second half where a much more conservative approach
is taken. It is rather obvious that a .0, .1 needs more stabilization than a .
12 or a .13

Just food for thought :-)

Even

(*) even bugfix ones, which is admidetly sometimes a bit overkill

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

pcav
Hi all,

Il 22/11/19 23:20, Even Rouault ha scritto:

> Or one might consider a mixed approach to have a good compromise of agility vs
> tighter control:
> - use time-based approach, as done currently, for non-LTR versions.
> - formally approve the release of LTR versions, and important engineering
> decisions that affect them, as there are the ones with the most user exposure.
> Sometime ago I also suggested to possibly consider 2 phases in a LTR life-
> cycle: first half where quite "aggressive" backporting is accepted (if it
> doesn't break API, etc..), second half where a much more conservative approach
> is taken. It is rather obvious that a .0, .1 needs more stabilization than a .
> 12 or a .13
>
> Just food for thought :-)
>
> Even
>
> (*) even bugfix ones, which is admidetly sometimes a bit overkill
>

Thanks everybody for the analysis and the proposals.
IMHO the LTR should be as stable as possible, so I agree that changing
things now it's unfortunate.
An alternative is to freeze it, and don't release further updates. Are
there important bugfixes preventing us to do so?
Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Tim Sutton-6
Hi Paolo



On 23 Nov 2019, at 08:05, Paolo Cavallini <[hidden email]> wrote:

Hi all,

Il 22/11/19 23:20, Even Rouault ha scritto:

Or one might consider a mixed approach to have a good compromise of agility vs
tighter control:
- use time-based approach, as done currently, for non-LTR versions.
- formally approve the release of LTR versions, and important engineering
decisions that affect them, as there are the ones with the most user exposure.
Sometime ago I also suggested to possibly consider 2 phases in a LTR life-
cycle: first half where quite "aggressive" backporting is accepted (if it
doesn't break API, etc..), second half where a much more conservative approach
is taken. It is rather obvious that a .0, .1 needs more stabilization than a .
12 or a .13

Just food for thought :-)

Even

(*) even bugfix ones, which is admidetly sometimes a bit overkill


Thanks everybody for the analysis and the proposals.
IMHO the LTR should be as stable as possible, so I agree that changing
things now it's unfortunate.
An alternative is to freeze it, and don't release further updates. Are
there important bugfixes preventing us to do so?

If I understand the situation properly, it is not possible to do this in the OSGEO4W framework (and maybe similar issue in Debian etc) since in OSGEO4W online installer there is only one proj, GDAL etc published and you cannot have e.g. QGIS  3.4.x depending on project 5 and QGIS 3.10.x depending on Proj 6 both both within the same OSGEO4W package tree. I think similar issue arises in Debian. I may be totally misreading the situation (I am sure Jürgen will correct me if I am :-) ).In the OSGEO4W side we would basically need to ask / fund Jürgen to support different packages for different versions of QGIS which I suspect may be a major pain in the butt.

For windows at least we could ‘freeze’ 3.4.x by pointing them to use the standalone installers but then we have an LTR that is not getting bug fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix releases of 3.4 before 3.10 becomes LTR, which is quite a long time to have our ’stable’ version out there with no bug fixes.

PR3.10.13.4.14 2019-11-29483
PR3.10.23.4.15 2019-12-20514
PR/FF3.10.33.4.163.112020-01-1735
LR/PR3.12.03.10.4 2020-02-2184
Regards

Tim




Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer









Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

I'd love to connect. Here's my calendar link to make finding time easy.


_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Sebastiaan Couwenberg
In reply to this post by pcav
On 11/23/19 9:05 AM, Paolo Cavallini wrote:
> IMHO the LTR should be as stable as possible, so I agree that changing
> things now it's unfortunate.

Changing dependencies are pretty much inevitable, LTR needs to adapt to
those or become unusable which defeats the purpose of LTR.

PROJ 6 & GDAL 3 are major changes and updating distributions when QGIS
3.10 supports them well is good time do migrate to them.

We're in the process to migrate to PROJ 6 and GDAL 3 in Debian as well,
we've had PROJ 6 in testing/unstable for a while now and GDAL 3
hopefully follows after the ongoing python3.8 transition. We'll have
QGIS 3.4 LTR built with that for now, until we switch the package to
3.10 once that becomes the new LTR and have the qgis package make full
use of PROJ 6 & GDAL 3.

Kind Regards,

Bas

--
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Borys Jurgiel-4
In reply to this post by Even Rouault-2
Thanks Even! I was mislead by the hard freeze and had an impression 3.10 is
going to be more stable than it actually was meant to be. I'm often not sure
what level of predictability we can expect from the official builds and your
response shreds more light on it.

And, first of all, big thanks to Nyall for working on fixing it!

Regards,
Borys

Dnia piątek, 22 listopada 2019 23:20:49 CET Even Rouault pisze:

> > QGIS LTR in Windows has also switched from proj 5.x and gdal 2.x to
> > proj 6.x and gdal 3.x which has resulted in some new bugs.
>
> I see 2 different situations:
>
> - 3.10 was intended to work with GDAL 3 & PROJ 6 (work started with 3.8 if I
> remember). OSGeo4W received a late upgrade to them because GRASS 7.8 wasn't
> ready yet for them (no offense to the GRASS team !), but devs have been
> able to work against GDAL 3 & PROJ 6, so that wasn't unknown territory.
> Probably that 3.10.0 could have been delayed while OSGeo4W hadn't switched
> to GDAL 3/ PROJ 6, but it wasn't clear when GRASS would be out. So overall
> how the situation was handled doesn't seem that bad for a release
> exercising new major dependencies.
>
> - I do agree that the switch to GDAL 3 & PROJ 6 for a near end-of-life 3.4
> LTR was unfortunate (but more than the timing, that does not seem
> appropriate for a release which was not designed originally to work with
> them) and I felt -0 on that.
>
> > It will be additional work for package managers and more resources, but it
> > will be good to have some guidelines on LTR and PRs dependencies for
> > packaging.
>
> Each LTR should ideally have its own set of dependencies, allowing
> controlled changes of them when needed (adopting a new bugfix version might
> be OK), and allowing the current version to use more aggressively new
> versions of the dependencies. That said, I'm not a packager, and don't know
> the effort & cost associated, but that's clearly an area where the project
> could/should invest. Similar situation is likely to happen again in the
> future.
>
> I'm also wondering if there shouldn't be a vote, from developers, PSC, bug
> triagers, testers or probably a mix of them forming a representative body,
> to adjudicate:
> - dependency upgrades
> - decision of releasing a new version (most other OSGeo projects I'm
> familiar with rely on a PSC formal vote to issue releases (*)).
>
> Or one might consider a mixed approach to have a good compromise of agility
> vs tighter control:
> - use time-based approach, as done currently, for non-LTR versions.
> - formally approve the release of LTR versions, and important engineering
> decisions that affect them, as there are the ones with the most user
> exposure. Sometime ago I also suggested to possibly consider 2 phases in a
> LTR life- cycle: first half where quite "aggressive" backporting is
> accepted (if it doesn't break API, etc..), second half where a much more
> conservative approach is taken. It is rather obvious that a .0, .1 needs
> more stabilization than a . 12 or a .13
>
> Just food for thought :-)
>
> Even
>
> (*) even bugfix ones, which is admidetly sometimes a bit overkill




_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

pcav
In reply to this post by Tim Sutton-6
Hi Tim,

Il 23/11/19 11:22, Tim Sutton ha scritto:


> For windows at least we could ‘freeze’ 3.4.x by pointing them to use the
> standalone installers but then we have an LTR that is not getting bug
> fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix
> releases of 3.4 before 3.10 becomes LTR, which is quite a long time to
> have our ’stable’ version out there with no bug fixes.

yes, that's what I meant. It's a kind of bad hack, so as Bas also
pointed out the proper solution is just to go ahead and fix the bugs.
I feel bad about people wasting their time to fix this compatibility bug
that will be useful just for a relatively short period, however.
Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Tim Sutton-6
Hi

We also have situations like this [1] where we are breaking the LTR with. Big bumps in GDAL/PROJ version.

Could we either a) invest QGIS bug fixing money into resolving these or b) adopt a more conservative approach to adopting new versions of libraries?


Regards

Tim

On 23 Nov 2019, at 11:23, Paolo Cavallini <[hidden email]> wrote:

Hi Tim,

Il 23/11/19 11:22, Tim Sutton ha scritto:


For windows at least we could ‘freeze’ 3.4.x by pointing them to use the
standalone installers but then we have an LTR that is not getting bug
fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix
releases of 3.4 before 3.10 becomes LTR, which is quite a long time to
have our ’stable’ version out there with no bug fixes.

yes, that's what I meant. It's a kind of bad hack, so as Bas also
pointed out the proper solution is just to go ahead and fix the bugs.
I feel bad about people wasting their time to fix this compatibility bug
that will be useful just for a relatively short period, however.
Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer









Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

I'd love to connect. Here's my calendar link to make finding time easy.


_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Nathan Woodrow
Hey,

I know it has been said before, but changing the GDAL/Proj version of the LTR versions is really a no go. It's a breaking change in terms of possible breakages and just increases risks on a version we need to stay stable.

We are communicating to users they should use the LTR because it's stable but changing the major dependencies under the hood with new versions breaks this trust and message.  It would be like changing the Qt version inside an LTR point release, we just can't do it because it's a full major change with massive impacts.   The risk of the LTR running on older stacks is seen as a net positive not a negative when it comes to stability for users.   The moment we break this trust we lose all the work past work we have put in to communicate this message, trust is hard to gain and easy to lose.

I'm not sure what the solution here is because I know OSGeo4w doesn't really support that kind of separation yet but if we have a way to fund this kind of work I would really like us to consider it (unsure the state of the budget but this one is critical IMO).  It's going to be an ongoing issue and something that will always have to be resolved with each LTR as it moves away from the nightlies with the  dependencies  changing.   I wish I had the skills and time to help fix this but at the moment all I can offer is my concern.

Regards,
Nathan

On Sat, Nov 23, 2019 at 10:26 PM Tim Sutton <[hidden email]> wrote:
Hi

We also have situations like this [1] where we are breaking the LTR with. Big bumps in GDAL/PROJ version.

Could we either a) invest QGIS bug fixing money into resolving these or b) adopt a more conservative approach to adopting new versions of libraries?


Regards

Tim

On 23 Nov 2019, at 11:23, Paolo Cavallini <[hidden email]> wrote:

Hi Tim,

Il 23/11/19 11:22, Tim Sutton ha scritto:


For windows at least we could ‘freeze’ 3.4.x by pointing them to use the
standalone installers but then we have an LTR that is not getting bug
fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix
releases of 3.4 before 3.10 becomes LTR, which is quite a long time to
have our ’stable’ version out there with no bug fixes.

yes, that's what I meant. It's a kind of bad hack, so as Bas also
pointed out the proper solution is just to go ahead and fix the bugs.
I feel bad about people wasting their time to fix this compatibility bug
that will be useful just for a relatively short period, however.
Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer









Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

I'd love to connect. Here's my calendar link to make finding time easy.

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

kimaidou
HI,
I totally agree with Nathan on this topic.
Regards,
Michaël

Le sam. 23 nov. 2019 à 13:37, Nathan Woodrow <[hidden email]> a écrit :
Hey,

I know it has been said before, but changing the GDAL/Proj version of the LTR versions is really a no go. It's a breaking change in terms of possible breakages and just increases risks on a version we need to stay stable.

We are communicating to users they should use the LTR because it's stable but changing the major dependencies under the hood with new versions breaks this trust and message.  It would be like changing the Qt version inside an LTR point release, we just can't do it because it's a full major change with massive impacts.   The risk of the LTR running on older stacks is seen as a net positive not a negative when it comes to stability for users.   The moment we break this trust we lose all the work past work we have put in to communicate this message, trust is hard to gain and easy to lose.

I'm not sure what the solution here is because I know OSGeo4w doesn't really support that kind of separation yet but if we have a way to fund this kind of work I would really like us to consider it (unsure the state of the budget but this one is critical IMO).  It's going to be an ongoing issue and something that will always have to be resolved with each LTR as it moves away from the nightlies with the  dependencies  changing.   I wish I had the skills and time to help fix this but at the moment all I can offer is my concern.

Regards,
Nathan

On Sat, Nov 23, 2019 at 10:26 PM Tim Sutton <[hidden email]> wrote:
Hi

We also have situations like this [1] where we are breaking the LTR with. Big bumps in GDAL/PROJ version.

Could we either a) invest QGIS bug fixing money into resolving these or b) adopt a more conservative approach to adopting new versions of libraries?


Regards

Tim

On 23 Nov 2019, at 11:23, Paolo Cavallini <[hidden email]> wrote:

Hi Tim,

Il 23/11/19 11:22, Tim Sutton ha scritto:


For windows at least we could ‘freeze’ 3.4.x by pointing them to use the
standalone installers but then we have an LTR that is not getting bug
fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix
releases of 3.4 before 3.10 becomes LTR, which is quite a long time to
have our ’stable’ version out there with no bug fixes.

yes, that's what I meant. It's a kind of bad hack, so as Bas also
pointed out the proper solution is just to go ahead and fix the bugs.
I feel bad about people wasting their time to fix this compatibility bug
that will be useful just for a relatively short period, however.
Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer









Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

I'd love to connect. Here's my calendar link to make finding time easy.

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Mathieu Pellerin
Add to what was said (which I don't disagree with per say). I think it's important to note that this GDAL3/PROJ6 transition was always going to be rocky (whether we applied it to 3.4 LTR or delayed it of 4 months when 3.10 LTR will replace 3.4. 

One reason being most core developers are on linux distributions, where a gdal3/proj6 environment isn't available out of the box. Coupled with the fact that our bank of daily / weekly testers is very small, we're left with mostly devs (on linux) testing builds (which we've established are vastly running gdal2).



On Sat, Nov 23, 2019, 21:35 kimaidou <[hidden email]> wrote:
HI,
I totally agree with Nathan on this topic.
Regards,
Michaël

Le sam. 23 nov. 2019 à 13:37, Nathan Woodrow <[hidden email]> a écrit :
Hey,

I know it has been said before, but changing the GDAL/Proj version of the LTR versions is really a no go. It's a breaking change in terms of possible breakages and just increases risks on a version we need to stay stable.

We are communicating to users they should use the LTR because it's stable but changing the major dependencies under the hood with new versions breaks this trust and message.  It would be like changing the Qt version inside an LTR point release, we just can't do it because it's a full major change with massive impacts.   The risk of the LTR running on older stacks is seen as a net positive not a negative when it comes to stability for users.   The moment we break this trust we lose all the work past work we have put in to communicate this message, trust is hard to gain and easy to lose.

I'm not sure what the solution here is because I know OSGeo4w doesn't really support that kind of separation yet but if we have a way to fund this kind of work I would really like us to consider it (unsure the state of the budget but this one is critical IMO).  It's going to be an ongoing issue and something that will always have to be resolved with each LTR as it moves away from the nightlies with the  dependencies  changing.   I wish I had the skills and time to help fix this but at the moment all I can offer is my concern.

Regards,
Nathan

On Sat, Nov 23, 2019 at 10:26 PM Tim Sutton <[hidden email]> wrote:
Hi

We also have situations like this [1] where we are breaking the LTR with. Big bumps in GDAL/PROJ version.

Could we either a) invest QGIS bug fixing money into resolving these or b) adopt a more conservative approach to adopting new versions of libraries?


Regards

Tim

On 23 Nov 2019, at 11:23, Paolo Cavallini <[hidden email]> wrote:

Hi Tim,

Il 23/11/19 11:22, Tim Sutton ha scritto:


For windows at least we could ‘freeze’ 3.4.x by pointing them to use the
standalone installers but then we have an LTR that is not getting bug
fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix
releases of 3.4 before 3.10 becomes LTR, which is quite a long time to
have our ’stable’ version out there with no bug fixes.

yes, that's what I meant. It's a kind of bad hack, so as Bas also
pointed out the proper solution is just to go ahead and fix the bugs.
I feel bad about people wasting their time to fix this compatibility bug
that will be useful just for a relatively short period, however.
Cheers.
--
Paolo Cavallini - www.faunalia.eu
QGIS.ORG Chair:
http://planet.qgis.org/planet/user/28/tag/qgis%20board/
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer









Tim Sutton

Co-founder: Kartoza
Ex Project chair: QGIS.org

Visit http://kartoza.com to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

I'd love to connect. Here's my calendar link to make finding time easy.

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Even Rouault-2
In reply to this post by pcav
On samedi 23 novembre 2019 12:23:12 CET Paolo Cavallini wrote:

> Hi Tim,
>
> Il 23/11/19 11:22, Tim Sutton ha scritto:
> > For windows at least we could ‘freeze’ 3.4.x by pointing them to use the
> > standalone installers but then we have an LTR that is not getting bug
> > fixes. As I read Jürgen’s roadmap [1] we still have 4 more bug fix
> > releases of 3.4 before 3.10 becomes LTR, which is quite a long time to
> > have our ’stable’ version out there with no bug fixes.
>
> yes, that's what I meant. It's a kind of bad hack, so as Bas also
> pointed out the proper solution is just to go ahead and fix the bugs.
> I feel bad about people wasting their time to fix this compatibility bug
> that will be useful just for a relatively short period, however.

I'm pretty sure there will be problems with CRS handling with QGIS 3.4 +
GDAL3/PROJ6 that will *not* be fixable, due to intended changes of behaviour
in GDAL3/PROJ6, particularly the export of CRS definition to PROJ strings,
that QGIS 3.4 still uses it, that is deprecated now and works only in a
degraded mode regarding export datum information (retrospectively, I should
probably have completely remove exportToProj4() to make that obvious). QGIS
3.4 to work properly with GDAL3/PROJ6 would require backporting all the PROJ6
specific work started with 3.8, which is another big no no.

For example just seing that srs.db in 3.4.13 package has for OSGB36:
parameters (String) = +proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717
+x_0=400000 +y_0=-100000 +ellps=airy +units=m +no_defs
where GDAL 2 based versions have an extra
+towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 which is
essential to make things work properly in a PROJ4 string oriented workflow as
used by 3.4. I'v just tried with a GPKG in WGS84 with a single point in
England, and a conversion of it to OSGB36 (with ogr2ogr of GDAL 3 or GDAL 2):
when loaded in 3.4.13 based on GDAL 3, the points are distant of 130 m (which
is the missing datum shift). Whereas when loaded with 3.4.13 based on GDAL 2
(or 3.10 with GDAL 3), they overlay properly.

So people dealing with datums != WGS84 should better stick with 3.4.12 on
Windows currently, if wanting to stay in the 3.4 series.

QGIS 3.4 combined with GDAL3/PROJ6 is a *design* bug, not a regular bug that
can be fixed.

I don't see why QGIS 3.4 on OSGeo4W using a GDAL 2.4.3 + PROJ 5.2
wouldn't be technically achievable. Isn't there a gdal and gdal-dev package in
it, several QGIS variants already ? So why not a gdal2_4 ? Well, I guess that
doing that now would require creating a grass package that depends on gdal2_4
etc. So yes, that might be painful to do at that stage. Dunno...

But for the future, I can imagine we could have:
qgis3_10 depending on gdal3 (or possibly even gdal3_0 ?) and proj6 and
qt_whatever_we_use_currently
And qgis3_12 could decide to upgrade one of its dependencies without impacting
qgis3_10.

Whatever a hot fix, or a long term solution (pun intended :-)) along the above
lines or better is chosen, it would seem to me as a very appropriate use of
funds of the project to investigate what is possible.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: LTR management [was Re: Delaying 3.10.1?]

Jürgen E. Fischer
In reply to this post by Even Rouault-2
Hi Even,

On Fri, 22. Nov 2019 at 23:20:49 +0100, Even Rouault wrote:
> > QGIS LTR in Windows has also switched from proj 5.x and gdal 2.x to
> > proj 6.x and gdal 3.x which has resulted in some new bugs.
 
> I see 2 different situations:
 
> - 3.10 was intended to work with GDAL 3 & PROJ 6 (work started with 3.8 if I
> remember). OSGeo4W received a late upgrade to them because GRASS 7.8 wasn't
> ready yet for them (no offense to the GRASS team !), but devs have been able
> to work against GDAL 3 & PROJ 6, so that wasn't unknown territory. Probably
> that 3.10.0 could have been delayed while OSGeo4W hadn't switched to GDAL 3/
> PROJ 6, but it wasn't clear when GRASS would be out.

More background:

* GDAL 3 requires PROJ 6
* QGIS 3.10 has optional features that require GDAL 3,
* GRASS uses OSGeo4W for dependencies on Windows,
* GRASS also contributes the GRASS packages to OSGeo4W,
* GRASS didn't support PROJ 6 & GDAL 3 until quite recently,
* GRASS was therefore blocking the PROJ 6 and GRAL 3 updates in OSGeo4W,
* GRASS' windows builds were not working for quite a while,
* I introduced nightly builds of GDAL 3 and PROJ 6 next to the regular GDAL and
  PROJ packages to OSGeo4W to be able to build the QGIS nightlies against GDAL
  3 - because the main GDAL was blocked and QGIS 3.10 with the optional features
  The release packages were still against GDAL 2.
* The GRASS' builds were only revived after I tried to build myself and
  contributed pull requests to fix their builds (and that still looked quite
  familiar, because it still resembled much what I contributed years ago).
* Those pull requests were also targeting on building GRASS with GDAL 3 and
  PROJ 6 in OSGeo4W and gave an other already pending GRASS pull request (not
  mine) that added GDAL 3 and PROJ 6 support a nudge and it was also merged
  into GRASS 7.8.1.
* When QGIS was released, GRASS was not yet released and there was no clear
  time frame on that.  So in OSGeo4W GDAL was not updated yet and hence QGIS
  was still built GDAL 2 and PROJ 6.
* QGIS 3.4 supports PROJ6 - apparently not many people tried it. The LTR
  nightlies in OSGeo4W were using it and Debian unstable also already provides
  PROJ6.  Not many to test - but also not none.
* People "asked" about the missing GDAL 3 features in QGIS 3.10.
* I prepared testing packages of the upgrades (GDAL 3, PROJ 6, QGIS 3.4 and
  3.10 and more) so that they could be used in the GRASS build, without meanwhile
  breaking the rest.
* Those were taken live along with the GRASS packages once those arrived.

> So overall how the situation was handled doesn't seem that bad for a release
> exercising new major dependencies.

Thank you.

> …

If i had to run this through a committee first, probably nothing of the above
had happend yet and I wouldn't have time left to actually do it.


Jürgen


PS: Windows CI status:
 * appveyor: still times out (after 4 years the limit of 1h still isn't enough)
 * azure pipelines: runs out of disk space (limit 10GB)

PPS: I also need something to punch ;)

--
Jürgen E. Fischer           norBIT GmbH             Tel. +49-4931-918175-31
Dipl.-Inf. (FH)             Rheinstraße 13          Fax. +49-4931-918175-50
Software Engineer           D-26506 Norden            https://www.norbit.de
QGIS release manager (PSC)  Germany                    IRC: jef on FreeNode

norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Juergen Fischer, Nils Kutscher HR: Amtsgericht Aurich HRB 100827
Datenschutzerklaerung: https://www.norbit.de/83/

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

signature.asc (844 bytes) Download Attachment
123