QtXml is no longer maintained in Qt5

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

QtXml is no longer maintained in Qt5

Matthias Kuhn
Hi,

As of Qt5, QtXml is no longer actively maintained [1]
PyQt5 does not even support it.

It will be a reasonably big undertaking to migrate any occurrence of
QDomDocument etc. to QXMLStreamWriter and QXMLStreamReader (our whole
project saving and loading engine is based on this). And when we do this
we will likely have to increase the major version number of QGIS.

In order not to introduce more work, please consider using
QXMLStreamWriter and QXMLStreamReader for new code you write that
involves parsing or producing XML. That does not concern working on code
that is based on code that is currently based on QtXml methods.

Thank you,
Matthias

[1] http://qt-project.org/doc/qt-5/qtxml-module.html#details

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Marco Hugentobler-4
Hi Matthias

>It will be a reasonably big undertaking to migrate any occurrence of
> QDomDocument etc. to QXMLStreamWriter and QXMLStreamReader


 From a quick look, it seems QXMLStreamReader is more like a SAX parser.
If so, it is not a full replacement for the DOM classes.

Regards,
Marco

On 03.12.2014 16:12, Matthias Kuhn wrote:

> Hi,
>
> As of Qt5, QtXml is no longer actively maintained [1]
> PyQt5 does not even support it.
>
> It will be a reasonably big undertaking to migrate any occurrence of
> QDomDocument etc. to QXMLStreamWriter and QXMLStreamReader (our whole
> project saving and loading engine is based on this). And when we do this
> we will likely have to increase the major version number of QGIS.
>
> In order not to introduce more work, please consider using
> QXMLStreamWriter and QXMLStreamReader for new code you write that
> involves parsing or producing XML. That does not concern working on code
> that is based on code that is currently based on QtXml methods.
>
> Thank you,
> Matthias
>
> [1] http://qt-project.org/doc/qt-5/qtxml-module.html#details
>
> _______________________________________________
> Qgis-developer mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/qgis-developer


--
Dr. Marco Hugentobler
Sourcepole -  Linux & Open Source Solutions
Weberstrasse 5, CH-8004 Zürich, Switzerland
[hidden email] http://www.sourcepole.ch
Technical Advisor QGIS Project Steering Committee

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Matthias Kuhn
Hi Marco,

I didn't look at it in detail and just based this recommendation on the
one from the QtXml documentation linked at the end of the original message.
Is there something in particular you think we will be missing? If so, it
might be good to start looking for alternative ways to go.

Regards,
Matthias

On 03.12.2014 16:24, Marco Hugentobler wrote:

> Hi Matthias
>
>> It will be a reasonably big undertaking to migrate any occurrence of
>> QDomDocument etc. to QXMLStreamWriter and QXMLStreamReader
>
>
> From a quick look, it seems QXMLStreamReader is more like a SAX
> parser. If so, it is not a full replacement for the DOM classes.
>
> Regards,
> Marco
>
> On 03.12.2014 16:12, Matthias Kuhn wrote:
>> Hi,
>>
>> As of Qt5, QtXml is no longer actively maintained [1]
>> PyQt5 does not even support it.
>>
>> It will be a reasonably big undertaking to migrate any occurrence of
>> QDomDocument etc. to QXMLStreamWriter and QXMLStreamReader (our whole
>> project saving and loading engine is based on this). And when we do this
>> we will likely have to increase the major version number of QGIS.
>>
>> In order not to introduce more work, please consider using
>> QXMLStreamWriter and QXMLStreamReader for new code you write that
>> involves parsing or producing XML. That does not concern working on code
>> that is based on code that is currently based on QtXml methods.
>>
>> Thank you,
>> Matthias
>>
>> [1] http://qt-project.org/doc/qt-5/qtxml-module.html#details
>>
>> _______________________________________________
>> Qgis-developer mailing list
>> [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Jürgen E. Fischer
Hi Matthias,

On Wed, 03. Dec 2014 at 16:46:11 +0100, Matthias Kuhn wrote:
> I didn't look at it in detail and just based this recommendation on the
> one from the QtXml documentation linked at the end of the original message.
> Is there something in particular you think we will be missing? If so, it
> might be good to start looking for alternative ways to go.

We could just keep using it and fork it if need be.  Any idea why it was
discontinued?


Jürgen

--
Jürgen E. Fischer           norBIT GmbH             Tel. +49-4931-918175-31
Dipl.-Inf. (FH)             Rheinstraße 13          Fax. +49-4931-918175-50
Software Engineer           D-26506 Norden             http://www.norbit.de
QGIS release manager (PSC)  Germany                    IRC: jef on FreeNode                        

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer

signature.asc (844 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Nyall Dawson
On 4 December 2014 at 19:25, Jürgen E. <[hidden email]> wrote:

> Hi Matthias,
>
> On Wed, 03. Dec 2014 at 16:46:11 +0100, Matthias Kuhn wrote:
>> I didn't look at it in detail and just based this recommendation on the
>> one from the QtXml documentation linked at the end of the original message.
>> Is there something in particular you think we will be missing? If so, it
>> might be good to start looking for alternative ways to go.
>
> We could just keep using it and fork it if need be.  Any idea why it was
> discontinued?
>

This link [1] suggests that it hasn't been discontinued, it's just
unmaintained, and isn't scheduled for removal. That doesn't solve the
PyQt5 problem though... It's also worth noting that QtXml is very
slow, which is noticeable on project load/save.


Nyall

[1] https://bugreports.qt-project.org/browse/QTBUG-32926
_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Matthias Kuhn
Thank you Jürgen and Nyall,


On 04.12.2014 09:44, Nyall Dawson wrote:
> On 4 December 2014 at 19:25, Jürgen E. <[hidden email]> wrote:
>> We could just keep using it and fork it if need be.  Any idea why it was
>> discontinued?
>>
> This link [1] suggests that it hasn't been discontinued, it's just
> unmaintained, and isn't scheduled for removal. That doesn't solve the
> PyQt5 problem though... It's also worth noting that QtXml is very
> slow, which is noticeable on project load/save.

I wrote a mail to the PyQt mailing list to ask about the status.

In general I think it is preferable to stay close to upstream as in the
long run mainenance effort is normally smaller.
But in this case a fork of QtXml (or its python bindings) may indeed be
a good option considering the amount of work and the API break involved.
For no apparent benefits and no clear migration path outlined.

Matthias

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

giohappy
In reply to this post by Nyall Dawson

QtXml is really, damn slow. PyQt with QtXml is worst then lxml, just to make a comparison. Switching to a streamlined xml parser would be advisable, but I know it would a deep refactoring for a lot of QGIS's code.
Maybe it should be considered within the next big refactorings (geometry, composer, etc.), but it won't be easy to find a sponsor specifically for this...

giovanni

Il 04/dic/2014 09:45 "Nyall Dawson" <[hidden email]> ha scritto:
On 4 December 2014 at 19:25, Jürgen E. <[hidden email]> wrote:
> Hi Matthias,
>
> On Wed, 03. Dec 2014 at 16:46:11 +0100, Matthias Kuhn wrote:
>> I didn't look at it in detail and just based this recommendation on the
>> one from the QtXml documentation linked at the end of the original message.
>> Is there something in particular you think we will be missing? If so, it
>> might be good to start looking for alternative ways to go.
>
> We could just keep using it and fork it if need be.  Any idea why it was
> discontinued?
>

This link [1] suggests that it hasn't been discontinued, it's just
unmaintained, and isn't scheduled for removal. That doesn't solve the
PyQt5 problem though... It's also worth noting that QtXml is very
slow, which is noticeable on project load/save.


Nyall

[1] https://bugreports.qt-project.org/browse/QTBUG-32926
_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Luca Manganelli
On Thu, Dec 4, 2014 at 10:04 AM, G. Allegri <[hidden email]> wrote:
> QtXml is really, damn slow. PyQt with QtXml is worst then lxml, just to make
> a comparison. Switching to a streamlined xml parser would be advisable, but
> I know it would a deep refactoring for a lot of QGIS's code.
> Maybe it should be considered within the next big refactorings (geometry,
> composer, etc.), but it won't be easy to find a sponsor specifically for
> this...

For a project at my organization, I used the standard python module
"XML.ElementTree" and it seems really good and fast. It's very easy to
read, modify, add elements...
_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Matthias Kuhn
In reply to this post by giohappy
On 04.12.2014 10:04, G. Allegri wrote:
> QtXml is really, damn slow. PyQt with QtXml is worst then lxml, just to
> make a comparison. Switching to a streamlined xml parser would be
> advisable, but I know it would a deep refactoring for a lot of QGIS's code.
> Maybe it should be considered within the next big refactorings (geometry,
> composer, etc.), but it won't be easy to find a sponsor specifically for
> this...
>
Do we have a real-world example of where we hit this performance issue?

Considering a sponsor: it might be packaged in a proposal of a native
project file format that also supports storing layers. That requires an
abstraction of the project file and has a new feature that usually makes
it easier to find funds.

Matthias

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Even Rouault-2
> Considering a sponsor: it might be packaged in a proposal of a native
> project file format that also supports storing layers.

You could potentially consider using a GeoPackage container as a base for that
:
- http://www.geopackage.org/spec/
- http://www.gdal.org/drv_geopackage.html

It would cover storing vector layers. The QGIS specific parts could be stored
in extension tables.

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Régis Haubourg
In reply to this post by Matthias Kuhn
Hi,
agreed it's awfully slow on large project save (about 1 min sometimes for about 200 layers)
+1 for refactoring qgs into a container able to store layers. That would give us a storing solution for memory layers, discussed previously.

Cheers
Régis
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Marco Hugentobler-4
In reply to this post by Nyall Dawson
>It's also worth noting that QtXml is very
>slow, which is noticeable on project load/save.


Is it really the xml parsing which is slow? I would have expected that
xml parsing only is fast and the things QGIS is doing on project load
are slow (e.g. restoring connections, creating print composers, ...).
Did you do some profiling / testing related to xml parsing speed?

Regards,
Marco

On 04.12.2014 09:44, Nyall Dawson wrote:

> On 4 December 2014 at 19:25, Jürgen E. <[hidden email]> wrote:
>> Hi Matthias,
>>
>> On Wed, 03. Dec 2014 at 16:46:11 +0100, Matthias Kuhn wrote:
>>> I didn't look at it in detail and just based this recommendation on the
>>> one from the QtXml documentation linked at the end of the original message.
>>> Is there something in particular you think we will be missing? If so, it
>>> might be good to start looking for alternative ways to go.
>> We could just keep using it and fork it if need be.  Any idea why it was
>> discontinued?
>>
> This link [1] suggests that it hasn't been discontinued, it's just
> unmaintained, and isn't scheduled for removal. That doesn't solve the
> PyQt5 problem though... It's also worth noting that QtXml is very
> slow, which is noticeable on project load/save.
>
>
> Nyall
>
> [1] https://bugreports.qt-project.org/browse/QTBUG-32926
> _______________________________________________
> Qgis-developer mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/qgis-developer


--
Dr. Marco Hugentobler
Sourcepole -  Linux & Open Source Solutions
Weberstrasse 5, CH-8004 Zürich, Switzerland
[hidden email] http://www.sourcepole.ch
Technical Advisor QGIS Project Steering Committee

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Luca Manganelli
On Thu, Dec 4, 2014 at 1:44 PM, Marco Hugentobler
<[hidden email]> wrote:
> Is it really the xml parsing which is slow? I would have expected that xml
> parsing only is fast and the things QGIS is doing on project load are slow
> (e.g. restoring connections, creating print composers, ...). Did you do some
> profiling / testing related to xml parsing speed?

I agree with Marco. The major slowdown here is when QGIS connects to
WMS/Postgis layers, not opening a simple XML file...

We could be very careful about false problems, as stated by Donald Knuth:

"We should forget about small efficiencies, say about 97% of the time:
premature optimization is the root of all evil"
_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Régis Haubourg
Slowness is painfull also on project save, and I guess this is mainly the xml issue there.
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Matthias Kuhn
In reply to this post by Matthias Kuhn
Hi all,

I'm happy to inform you that Phil (main PyQt maintainer) just announced
that QtXml is going to be reincluded in the PyQt5 bindings.
You may therefore happily continue to use QDomDocument and friends as
much as you like and ignore my original request.

Sorry for the noise
Matthias


On 12/03/2014 04:12 PM, Matthias Kuhn wrote:

> Hi,
>
> As of Qt5, QtXml is no longer actively maintained [1]
> PyQt5 does not even support it.
>
> It will be a reasonably big undertaking to migrate any occurrence of
> QDomDocument etc. to QXMLStreamWriter and QXMLStreamReader (our whole
> project saving and loading engine is based on this). And when we do this
> we will likely have to increase the major version number of QGIS.
>
> In order not to introduce more work, please consider using
> QXMLStreamWriter and QXMLStreamReader for new code you write that
> involves parsing or producing XML. That does not concern working on code
> that is based on code that is currently based on QtXml methods.
>
> Thank you,
> Matthias
>
> [1] http://qt-project.org/doc/qt-5/qtxml-module.html#details
>
> _______________________________________________
> Qgis-developer mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QtXml is no longer maintained in Qt5

Matthias Kuhn
In reply to this post by Régis Haubourg
Hi Régis,

On 12/04/2014 05:18 PM, Régis Haubourg wrote:
> Slowness is painfull also on project save, and I guess this is mainly the xml
> issue there.
>
>
I'm with Marco there. On project save there is also a lot of code
involved apart from XML and to attribute performance responsibility we
need good evidence first.

If there is a performance issue that you would like to tackle, I would
start with profiling this process and only after decide where the best
benefit can be achieved.

Matthias
_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer