python function to upgrade qgs datasources?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

python function to upgrade qgs datasources?

Régis Haubourg
Hi All,
We are preparing a great postgres migration here, and to make it transparent, we have given virtual aliases to our server.
User's QGS file refer today to hard coded IP or aliases (172.16.9.147 or vslin147).

We have been migrating most qgs using bash script with grep and sed, but we seem to touch the limits of such approaches (error handling, disconnected disks.. editions pending ., some are zipped )

My question, do you think it is possible to have a python function (plugin or user function), that will catch qgs before loading it, and that will substitute the host with the new value, before qgis loads the datasource ?

Any idea?

Cheers

Régis
Reply | Threaded
Open this post in threaded view
|

Re: python function to upgrade qgs datasources?

Andreas Neumann-4
Hi Régis,

I can't help you with the Python script to process your files, but I
have a suggestion:

Use a PostgreSQL service instead of hard-coded IP addresses, names or
aliases.

In such a scenario you can easily change between a production and test
server and more easily migrate to a new machine. Also, on a laptop you
can point the service to localhost (for mobile laptops) and within your
organizations LAN you can point to the production server. All without
touching the .qgs project file.

We in Uster are using PostgreSQL service names for several years now and
I like them. Unfortunately the concept of PostgreSQL services is not
very well-known among the PostgreSQL users.

See http://www.postgresql.org/docs/current/static/libpq-pgservice.html
and
http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS

Andreas

Am 01.07.2014 14:01, schrieb Régis Haubourg:

> Hi All,
> We are preparing a great postgres migration here, and to make it
> transparent, we have given virtual aliases to our server.
> User's QGS file refer today to hard coded IP or aliases (172.16.9.147 or
> vslin147).
>
> We have been migrating most qgs using bash script with grep and sed, but we
> seem to touch the limits of such approaches (error handling, disconnected
> disks.. editions pending ., some are zipped )
>
> My question, do you think it is possible to have a python function (plugin
> or user function), that will catch qgs before loading it, and that will
> substitute the host with the new value, before qgis loads the datasource ?
>
> Any idea?
>
> Cheers
>
> Régis
>
>
>
> --
> View this message in context: http://osgeo-org.1560.x6.nabble.com/python-function-to-upgrade-qgs-datasources-tp5149022.html
> Sent from the Quantum GIS - Developer mailing list archive at Nabble.com.
> _______________________________________________
> Qgis-developer mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: python function to upgrade qgs datasources?

Régis Haubourg
Great, thanks Andrea, I knew that system coming from Oracle, but we didn't find much doc on it so we did not focused a lot.
Next migration will upgrade to using services for sure..
By now, I still have to deal with preexisting documents.

switch as been done now, iut works anyway ;) let's us materialized views and foreign data wrappers !
Reply | Threaded
Open this post in threaded view
|

Re: python function to upgrade qgs datasources?

Régis Haubourg
In reply to this post by Andreas Neumann-4
Andreas Neumann wrote
Hi Régis,

I can't help you with the Python script to process your files, but I
have a suggestion:

Use a PostgreSQL service instead of hard-coded IP addresses, names or
aliases.

In such a scenario you can easily change between a production and test
server and more easily migrate to a new machine. Also, on a laptop you
can point the service to localhost (for mobile laptops) and within your
organizations LAN you can point to the production server. All without
touching the .qgs project file.

We in Uster are using PostgreSQL service names for several years now and
I like them. Unfortunately the concept of PostgreSQL services is not
very well-known among the PostgreSQL users.

See http://www.postgresql.org/docs/current/static/libpq-pgservice.html
and
http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS

Andreas
Hi Andreas,
I'm struggling to have connection using service work here on windows.
I tried putting a pg_service.conf file in AppData/postgres dir, or on a location defined by environnement variable with no success.

The content seems ok , but client (pgadmin, qgis) do not locate that file correctly. Do you have any hint for us? Pg doc is light, and the web is almost totally empty on that.

Régis
Reply | Threaded
Open this post in threaded view
|

Re: python function to upgrade qgs datasources?

Jürgen E. Fischer
Hi Régis,

On Wed, 20. Aug 2014 at 08:57:09 -0700, Régis Haubourg wrote:
> I'm struggling to have connection using service work here on windows.
> I tried putting a pg_service.conf file in AppData/postgres dir, or on a
> location defined by environnement variable with no success.

procmon is your friend.  The file is called .pg_service.conf - and apparently
libpq expects it to be in unix format (ie. only LF as line ending and not CRLF)

Jürgen

--
Jürgen E. Fischer           norBIT GmbH             Tel. +49-4931-918175-31
Dipl.-Inf. (FH)             Rheinstraße 13          Fax. +49-4931-918175-50
Software Engineer           D-26506 Norden             http://www.norbit.de
QGIS release manager (PSC)  Germany                    IRC: jef on FreeNode                        

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: python function to upgrade qgs datasources?

Régis Haubourg
It works!

you saved me again Jürgen!

I'm not totally clear yet on how to use procmon for that.. (just another tip?)

Cheers
Reply | Threaded
Open this post in threaded view
|

Re: python function to upgrade qgs datasources?

Jürgen E. Fischer
Hi Régis,

On Thu, 21. Aug 2014 at 01:07:41 -0700, Régis Haubourg wrote:
> It works!
>
> you saved me again Jürgen!
>
> I'm not totally clear yet on how to use procmon for that.. (just another
> tip?)

Download procmon[1], run it and filter for process name qgis(-dev)-bin.exe and
try a connection using a postgresql service.  QGIS (or better put libpq linked
into it) will then look for .pg_service.conf in what seems to be
%APPDATA%\postgresql.

On second thought the unix file ending thing might be due to the fact that we
set the runtime library to binary mode, which libpq might not expect.  Maybe
browser doesn't do that.


Jürgen

[1] http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

--
Jürgen E. Fischer           norBIT GmbH             Tel. +49-4931-918175-31
Dipl.-Inf. (FH)             Rheinstraße 13          Fax. +49-4931-918175-50
Software Engineer           D-26506 Norden             http://www.norbit.de
QGIS release manager (PSC)  Germany                    IRC: jef on FreeNode                        

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
Qgis-developer mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/qgis-developer