QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
Hi devs

There seems to be a performance issue in QGIS Server. I tried to compile different versions and got the following results :

QGIS master df19fe5e04b0c2ae8f1b13000099715559e6bd50 --> works as expected = pretty fast ;)

QGIS today's master -> a lot of performance issues. I get a lot of the following errors: http://paste.ubuntu.com/5690303/

So there is a commit in between which raises the issue. I can go on the process of compiling every commit to find the one which triggers the issue, unless someone knows

Anyone confirms ?

Regards
Michael

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
I checked other commits :

pyqgis-console] allow to disable automatic parentheses insertion from settings (commit 942d6bfa5d5e9cc7471efcd34717613fa6a8b7db )  --> performance issues

Fix wmsserver: Do not manipulate maplayerregistry directly (commit df19fe5e04b0c2ae8f1b13000099715559e6bd50 ) --> Good perfs

So the problem is in between. I continue testing commits

++


2013/5/22 kimaidou <[hidden email]>
Hi devs

There seems to be a performance issue in QGIS Server. I tried to compile different versions and got the following results :

QGIS master df19fe5e04b0c2ae8f1b13000099715559e6bd50 --> works as expected = pretty fast ;)

QGIS today's master -> a lot of performance issues. I get a lot of the following errors: http://paste.ubuntu.com/5690303/

So there is a commit in between which raises the issue. I can go on the process of compiling every commit to find the one which triggers the issue, unless someone knows

Anyone confirms ?

Regards
Michael


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

Jürgen E. Fischer
In reply to this post by kimaidou
Hi Michael,

On Wed, 22. May 2013 at 16:14:50 +0200, kimaidou wrote:
> So there is a commit in between which raises the issue. I can go on the
> process of compiling every commit to find the one which triggers the
> issue, unless someone knows

I don't.  But git bisect will probably take you there pretty quick. ;)


Jürgen

--
Jürgen E. Fischer         norBIT GmbH               Tel. +49-4931-918175-31
Dipl.-Inf. (FH)           Rheinstraße 13            Fax. +49-4931-918175-50
Software Engineer         D-26506 Norden               http://www.norbit.de
committ(ed|ing) to Quantum GIS                         IRC: jef on FreeNode                        

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
Thanks Jürgen for this great tip

I almost trapped the "vicious" commit !

Following commit has BAD Server perfs
Fix wmsserver: Do not manipulate maplayerregistry directly df19fe5e04b0c2ae8f1b13000099715559e6bd50

Following commit has GOOD Server perfs
Merge pull request #591 from ccrook/delimited_text_to_core … 91f3ceab26013d6c97a0047c344f5e2570df34c3


I tried to compile commits between these 2 above but cannot due to compilation errors.

I let users with better C++ skills and good QGIS Server knowledge find the problem

Thanks for any further help regarding this regression

Michael



2013/5/22 Jürgen E. <[hidden email]>
Hi Michael,

On Wed, 22. May 2013 at 16:14:50 +0200, kimaidou wrote:
> So there is a commit in between which raises the issue. I can go on the
> process of compiling every commit to find the one which triggers the
> issue, unless someone knows

I don't.  But git bisect will probably take you there pretty quick. ;)


Jürgen

--
Jürgen E. Fischer         norBIT GmbH               Tel. +49-4931-918175-31
Dipl.-Inf. (FH)           Rheinstraße 13            Fax. +49-4931-918175-50
Software Engineer         D-26506 Norden               http://www.norbit.de
committ(ed|ing) to Quantum GIS                         IRC: jef on FreeNode

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
Here is the result of git log between the 2 commits

df19fe5e04b0c2ae8f1b13000099715559e6bd50 Fix wmsserver: Do not manipulate maplayerregistry directly   ---> BAD
55556fcdc05fde836cb93985a9979636c999ec32 Use a cosmetic pen for snap lines to prevent lines scaling (fix #7524)
d88872806ff6d380e396dd19fabdd619566e1ad7 Fix generation of empty layer/style entries
35de29df9bea0a8d36d54aec677df054d7cd37ba Disallow changing of map layer registry directly QgsMapLayerRegistry::mapLayers() returned reference is
91f3ceab26013d6c97a0047c344f5e2570df34c3 Merge pull request #591 from ccrook/delimited_text_to_core   --> OK



2013/5/22 kimaidou <[hidden email]>
Thanks Jürgen for this great tip

I almost trapped the "vicious" commit !

Following commit has BAD Server perfs
Fix wmsserver: Do not manipulate maplayerregistry directly df19fe5e04b0c2ae8f1b13000099715559e6bd50

Following commit has GOOD Server perfs
Merge pull request #591 from ccrook/delimited_text_to_core … 91f3ceab26013d6c97a0047c344f5e2570df34c3


I tried to compile commits between these 2 above but cannot due to compilation errors.

I let users with better C++ skills and good QGIS Server knowledge find the problem

Thanks for any further help regarding this regression

Michael



2013/5/22 Jürgen E. <[hidden email]>

Hi Michael,

On Wed, 22. May 2013 at 16:14:50 +0200, kimaidou wrote:
> So there is a commit in between which raises the issue. I can go on the
> process of compiling every commit to find the one which triggers the
> issue, unless someone knows

I don't.  But git bisect will probably take you there pretty quick. ;)


Jürgen

--
Jürgen E. Fischer         norBIT GmbH               Tel. +49-4931-918175-31
Dipl.-Inf. (FH)           Rheinstraße 13            Fax. +49-4931-918175-50
Software Engineer         D-26506 Norden               http://www.norbit.de
committ(ed|ing) to Quantum GIS                         IRC: jef on FreeNode

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer



_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

Matthias Kuhn
In reply to this post by kimaidou
Hi Michael,

It's probably the commit you identified:

Fix wmsserver: Do not manipulate maplayerregistry directly
df19fe5e04b0c2ae8f1b13000099715559e6bd50

I'm almost sure if you revert this one and commit
35de29df9bea0a8d36d54aec677df054d7cd37ba it will run fine.

I'll have a look and revert these in case I don't find a solution to
get around this.

As I don't have a mapserver running here, could you assist with testing
if I have a possible fix ready?

Regards,
Matthias

On Mit 22 Mai 2013 17:51:01 CEST, kimaidou wrote:

> Thanks Jürgen for this great tip
>
> I almost trapped the "vicious" commit !
>
> Following commit has BAD Server perfs
> Fix wmsserver: Do not manipulate maplayerregistry directly
> df19fe5e04b0c2ae8f1b13000099715559e6bd50
>
> Following commit has GOOD Server perfs
> Merge pull request #591 from ccrook/delimited_text_to_core …
> 91f3ceab26013d6c97a0047c344f5e2570df34c3
>
>
> I tried to compile commits between these 2 above but cannot due to
> compilation errors.
>
> I let users with better C++ skills and good QGIS Server knowledge find
> the problem
>
> Thanks for any further help regarding this regression
>
> Michael
>
>
>
> 2013/5/22 Jürgen E. <[hidden email] <mailto:[hidden email]>>
>
>     Hi Michael,
>
>     On Wed, 22. May 2013 at 16:14:50 +0200, kimaidou wrote:
>     > So there is a commit in between which raises the issue. I can go
>     on the
>     > process of compiling every commit to find the one which triggers the
>     > issue, unless someone knows
>
>     I don't.  But git bisect will probably take you there pretty quick. ;)
>
>
>     Jürgen
>
>     --
>     Jürgen E. Fischer         norBIT GmbH               Tel.
>     +49-4931-918175-31
>     Dipl.-Inf. (FH)           Rheinstraße 13            Fax.
>     +49-4931-918175-50
>     Software Engineer         D-26506 Norden
>     http://www.norbit.de
>     committ(ed|ing) to Quantum GIS                         IRC: jef on
>     FreeNode
>
>     --
>     norBIT Gesellschaft fuer Unternehmensberatung und
>     Informationssysteme mbH
>     Rheinstrasse 13, 26506 Norden
>     GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502
>
>     _______________________________________________
>     Qgis-developer mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
>
> _______________________________________________
> Qgis-developer mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/qgis-developer


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou



2013/5/22 Matthias Kuhn <[hidden email]>
Hi Michael,

It's probably the commit you identified:

Fix wmsserver: Do not manipulate maplayerregistry directly
df19fe5e04b0c2ae8f1b13000099715559e6bd50


I think so too.
 

I'm almost sure if you revert this one and commit
35de29df9bea0a8d36d54aec677df054d7cd37ba it will run fine.

I'll have a look and revert these in case I don't find a solution to
get around this.

As I don't have a mapserver running here, could you assist with testing
if I have a possible fix ready?

Yes, sure I will !

I would like to try reverting only this commit, but I will need help here. Your idea is to revert this commit, then checkout into latest today's commit ? Could you help providing relative git commands ?

Thanks

 

Regards,
Matthias

On Mit


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

Matthias Kuhn
On top of current master (so checked out most recent commit with bad
performance):
git revert df19fe5e04b0c2ae8f1b13000099715559e6bd50
git revert 35de29df9bea0a8d36d54aec677df054d7cd37ba

One more question: Do you have any plugins running? (Is this even
supported by server)?

On Mit 22 Mai 2013 18:01:27 CEST, kimaidou wrote:

>
>
>
> 2013/5/22 Matthias Kuhn <[hidden email]
> <mailto:[hidden email]>>
>
>     Hi Michael,
>
>     It's probably the commit you identified:
>
>     Fix wmsserver: Do not manipulate maplayerregistry directly
>     df19fe5e04b0c2ae8f1b13000099715559e6bd50
>
>
>
> I think so too.
>
>
>
>     I'm almost sure if you revert this one and commit
>     35de29df9bea0a8d36d54aec677df054d7cd37ba it will run fine.
>
>     I'll have a look and revert these in case I don't find a solution to
>     get around this.
>
>     As I don't have a mapserver running here, could you assist with
>     testing
>     if I have a possible fix ready?
>
>
> Yes, sure I will !
>
> I would like to try reverting only this commit, but I will need help
> here. Your idea is to revert this commit, then checkout into latest
> today's commit ? Could you help providing relative git commands ?
>
> Thanks
>
>
>
>
>     Regards,
>     Matthias
>
>     On Mit
>
>


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
Answers below

2013/5/22 Matthias Kuhn <[hidden email]>
On top of current master (so checked out most recent commit with bad
performance):
git revert df19fe5e04b0c2ae8f1b13000099715559e6bd50
git revert 35de29df9bea0a8d36d54aec677df054d7cd37ba


Ok. So I reverted these 2 commits. I am now runnin the compilation (it will last approximatively 10 minutes)
I will tell you if the regression occurs or not after this changes

 
One more question: Do you have any plugins running? (Is this even
supported by server)?

QGIS Server does not support plugins, so I think this won't change anything.

Thanks for your help

Michael
 

On Mit 22 Mai 2013 18:01:27 CEST, kimaidou wrote:
>
>
>
> 2013/5/22 Matthias Kuhn <[hidden email]
> <mailto:[hidden email]>>
>
>     Hi Michael,
>
>     It's probably the commit you identified:
>
>     Fix wmsserver: Do not manipulate maplayerregistry directly
>     df19fe5e04b0c2ae8f1b13000099715559e6bd50
>
>
>
> I think so too.
>
>
>
>     I'm almost sure if you revert this one and commit
>     35de29df9bea0a8d36d54aec677df054d7cd37ba it will run fine.
>
>     I'll have a look and revert these in case I don't find a solution to
>     get around this.
>
>     As I don't have a mapserver running here, could you assist with
>     testing
>     if I have a possible fix ready?
>
>
> Yes, sure I will !
>
> I would like to try reverting only this commit, but I will need help
> here. Your idea is to revert this commit, then checkout into latest
> today's commit ? Could you help providing relative git commands ?
>
> Thanks
>
>
>
>
>     Regards,
>     Matthias
>
>     On Mit
>
>




_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
Matthias,

Reverting the 2 commits above latests master works -> regression is gone, and QGIS Server is as fast as usual.

Let me know if you want me to try some branch of yours for testing QGIS Server.

Cheers
Michael

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

Matthias Kuhn
The pull request [1] solves this. I don't really like this fix, but I
don't know the mapserver source well enough to provide a better fix.

Marco, could you have a look at the pull request and give a short
feedback. Further discussion could also happen there.

Regards,
Matthias

[1] https://github.com/qgis/Quantum-GIS/pull/604


On Mit 22 Mai 2013 18:21:12 CEST, kimaidou wrote:

> Matthias,
>
> Reverting the 2 commits above latests master works -> regression is
> gone, and QGIS Server is as fast as usual.
>
> Let me know if you want me to try some branch of yours for testing
> QGIS Server.
>
> Cheers
> Michael


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: QGIS Server - performance issue in master since one week (after df19fe5e04b0c2ae8f1b13000099715559e6bd50 )

kimaidou
Thanks Matthias for helping me to solve this out.
I subscribed to the Git pull request page.

Good evening


2013/5/22 Matthias Kuhn <[hidden email]>
The pull request [1] solves this. I don't really like this fix, but I
don't know the mapserver source well enough to provide a better fix.

Marco, could you have a look at the pull request and give a short
feedback. Further discussion could also happen there.

Regards,
Matthias

[1] https://github.com/qgis/Quantum-GIS/pull/604


On Mit 22 Mai 2013 18:21:12 CEST, kimaidou wrote:
> Matthias,
>
> Reverting the 2 commits above latests master works -> regression is
> gone, and QGIS Server is as fast as usual.
>
> Let me know if you want me to try some branch of yours for testing
> QGIS Server.
>
> Cheers
> Michael


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer


_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer