A snap for PostGis

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

A snap for PostGis

Julia Palandri
Hi everyone!

I’m Julia. I work at Canonical on the engineering team building Ubuntu and Snapcraft [1].


We’re working on snaps, a platform to enable ISVs to address a considerably wider audience while retaining full control over updates. Snaps are “write-once, run on many distro versions”, including both Ubuntu 14.04, 16.04, and many others (http://snapcraft.io/docs/core/install) - they make dependency problems a thing of the past, and offer early adopters the possibility to try new features as soon as they’re committed to trunk, all with minimum effort from your side (the hosting on our side is complimentary ;)).


Snaps help also to handle upgrades: they’re done transparently for users, which is convenient for them and for you (no users stuck in a super old version no longer supported, no work need to be done by them). And, in case anything bad happens (because… bad things sometimes happen :D) there will always be the chance of rollback available.


I first wrote to a member of the PSC, who asked me to write here instead.

I was wondering if you’d be willing to discuss some ideas on how we can help deliver PostGis to many more users?


Cheers,


[1] http://snapcraft.io/


--

Julia

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Paul Ramsey
Hi Julia,
I have a number of questions, none of which really bear on us actually doing the work to make a snap :)
- how is this different from other Ubuntu/Canonical initiatives like Juju and Charms?
- is this a Canonical/Linux flavour of Docker (lighter, better, Linux-only)?
- PostGIS is a run-time loadable DLL for PgSQL. Can the PgSQL snap really reach into another snap (PostGIS) and load the .so from there? Seems like we rather heavily break the isolation model.
- How does the PgSQL snap deal with major version updates? It's all very well going from 9.4.1 to 9.4.2, but 9.4 to 9.5 can be a huge deal and is fraught with dangers. Just run pg_upgrade and cross your fingers? We'd have to run SQL directly on the database, and potentially with quite high privilege (foreach db with postgis, run 'alter extension postgis update to '...'"
Thanks!

P


On Mon, Nov 28, 2016 at 9:08 AM, Julia Palandri <[hidden email]> wrote:
Hi everyone!

I’m Julia. I work at Canonical on the engineering team building Ubuntu and Snapcraft [1].


We’re working on snaps, a platform to enable ISVs to address a considerably wider audience while retaining full control over updates. Snaps are “write-once, run on many distro versions”, including both Ubuntu 14.04, 16.04, and many others (http://snapcraft.io/docs/core/install) - they make dependency problems a thing of the past, and offer early adopters the possibility to try new features as soon as they’re committed to trunk, all with minimum effort from your side (the hosting on our side is complimentary ;)).


Snaps help also to handle upgrades: they’re done transparently for users, which is convenient for them and for you (no users stuck in a super old version no longer supported, no work need to be done by them). And, in case anything bad happens (because… bad things sometimes happen :D) there will always be the chance of rollback available.


I first wrote to a member of the PSC, who asked me to write here instead.

I was wondering if you’d be willing to discuss some ideas on how we can help deliver PostGis to many more users?


Cheers,


[1] http://snapcraft.io/


--

Julia

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Sandro Santilli-4
In reply to this post by Julia Palandri
On Mon, Nov 28, 2016 at 06:08:22PM +0100, Julia Palandri wrote:

> I was wondering if you’d be willing to discuss some ideas on how we can
> help deliver PostGis to many more users?

Can "snap" users choose the repository from which to download snaps ?

--strk;
_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Julia Palandri
In reply to this post by Paul Ramsey
Hi Paul!

Thanks for your questions! I'll try to address them one by one:

On Mon, Nov 28, 2016 at 7:20 PM Paul Ramsey <[hidden email]> wrote:
Hi Julia,
I have a number of questions, none of which really bear on us actually doing the work to make a snap :)
- how is this different from other Ubuntu/Canonical initiatives like Juju and Charms?

Well, Juju is an orchestration service. It's focused rather in the "forest" than in the "tree" :)
Snaps don't know about the rest of the nodes in your cluster, for example, while Juju (or other orchestration software) will, and could tell your db snap in a node it can replicate to other nodes.
Juju and Snaps are not bound together, you can use one without the other. 
 
- is this a Canonical/Linux flavour of Docker (lighter, better, Linux-only)?

Hm, well, snaps are 'smaller' than docker: it allows (rather, constrains) app confinement, but doesn't provide OS confinement. 
You can, however, have them interact: you can use snaps inside Docker containers. Or you could use Ubuntu Core and the Docker snap to have a really secure host for your Docker containers.
 
- PostGIS is a run-time loadable DLL for PgSQL. Can the PgSQL snap really reach into another snap (PostGIS) and load the .so from there? Seems like we rather heavily break the isolation model.

If PostGIS is mostly used with a dedicated Postgres, it might be a good idea to bundle Postgres in the PostGIS snap.

If instead PostGIS is thought to work on any preinstalled postgres database (snap or not), we'd probably be talking about a new snap "interface" that would let your PostGIS snap copy its .so outside of its confinement, into the Postgres directory.

Visually, If the confinement around each snap could be a box, "interfaces" change the shape of that box.
 
- How does the PgSQL snap deal with major version updates? It's all very well going from 9.4.1 to 9.4.2, but 9.4 to 9.5 can be a huge deal and is fraught with dangers. Just run pg_upgrade and cross your fingers? We'd have to run SQL directly on the database, and potentially with quite high privilege (foreach db with postgis, run 'alter extension postgis update to '...'"

At the moment there are different snaps for each major version available (snap find postgres shows this), all of these in the stable channel. We are, however, working towards having smarter channels. Right now there is edge, which you can update with every commit to trunk. There's also beta, candidate, and of course stable.
We're aiming at having 9.6/stable, 9.6/candidate, 9.6/beta, 9.6/edge, 9.5/stable, 9.5/candidate, and so on, and also a "magic" latest family: latest/edge, latest/beta, etc. latest'stable would be the version you get when you ask the system to install any snap, without manually specifying the major version and flavour.

Hope to have helped!

Best,
 
Thanks!

P


On Mon, Nov 28, 2016 at 9:08 AM, Julia Palandri <[hidden email]> wrote:
Hi everyone!

I’m Julia. I work at Canonical on the engineering team building Ubuntu and Snapcraft [1].


We’re working on snaps, a platform to enable ISVs to address a considerably wider audience while retaining full control over updates. Snaps are “write-once, run on many distro versions”, including both Ubuntu 14.04, 16.04, and many others (http://snapcraft.io/docs/core/install) - they make dependency problems a thing of the past, and offer early adopters the possibility to try new features as soon as they’re committed to trunk, all with minimum effort from your side (the hosting on our side is complimentary ;)).


Snaps help also to handle upgrades: they’re done transparently for users, which is convenient for them and for you (no users stuck in a super old version no longer supported, no work need to be done by them). And, in case anything bad happens (because… bad things sometimes happen :D) there will always be the chance of rollback available.


I first wrote to a member of the PSC, who asked me to write here instead.

I was wondering if you’d be willing to discuss some ideas on how we can help deliver PostGis to many more users?


Cheers,


[1] http://snapcraft.io/


--

Julia

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Darafei "Komяpa" Praliaskouski

At the moment there are different snaps for each major version available (snap find postgres shows this), all of these in the stable channel. We are, however, working towards having smarter channels. Right now there is edge, which you can update with every commit to trunk. There's also beta, candidate, and of course stable.
We're aiming at having 9.6/stable, 9.6/candidate, 9.6/beta, 9.6/edge, 9.5/stable, 9.5/candidate, and so on, and also a "magic" latest family: latest/edge, latest/beta, etc. latest'stable would be the version you get when you ask the system to install any snap, without manually specifying the major version and flavour.

This describes install, but not upgrade process.

If you upgrade by just replacing binaries, you should keep in mind postgres 9.6 can't work with database created in postgres 9.4 - so, you need pg_upgrade (+ several manual steps) to perform the upgrade.

If you install postgis, you have a postgis binary for each version of postgis for each version of postgres. After you connect to postgres database, you can switch the version of extension used, and several running simultaneous queries can be using different versions of postgis.

Packaging postgis+postgres in single package has a disadvantage - you can't use more than two extensions at the same time in that scheme.

How is it going to be handled?

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Julia Palandri
In reply to this post by Sandro Santilli-4
Hi Sandro,

On Tue, Nov 29, 2016 at 12:03 AM Sandro Santilli <[hidden email]> wrote:
On Mon, Nov 28, 2016 at 06:08:22PM +0100, Julia Palandri wrote:

> I was wondering if you’d be willing to discuss some ideas on how we can
> help deliver PostGis to many more users?

Can "snap" users choose the repository from which to download snaps ?


Yes.
Publishers may publish in as many stores as store owners given them permission too. In addition, you can create you own store: http://snapcraft.io/docs/core/store and https://github.com/noise/snapstore/ should provide some insight.
Of course, we'd like to see upstream snaps in the Ubuntu store even if you decide to also create a separate one.

Best,

Julia

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Julia Palandri
In reply to this post by Darafei "Komяpa" Praliaskouski
Hi,

On Tue, Nov 29, 2016 at 1:10 PM Darafei "Komяpa" Praliaskouski <[hidden email]> wrote:

At the moment there are different snaps for each major version available (snap find postgres shows this), all of these in the stable channel. We are, however, working towards having smarter channels. Right now there is edge, which you can update with every commit to trunk. There's also beta, candidate, and of course stable.
We're aiming at having 9.6/stable, 9.6/candidate, 9.6/beta, 9.6/edge, 9.5/stable, 9.5/candidate, and so on, and also a "magic" latest family: latest/edge, latest/beta, etc. latest'stable would be the version you get when you ask the system to install any snap, without manually specifying the major version and flavour.

This describes install, but not upgrade process.


Well, rather, I was trying to explain that with different "series" for each major version, updates should be handled ok.
 
If you upgrade by just replacing binaries, you should keep in mind postgres 9.6 can't work with database created in postgres 9.4 - so, you need pg_upgrade (+ several manual steps) to perform the upgrade.


Right. So, I've been talking with the people that has been helping with the postgres snap, and they told that maybe the content interface might be able to give a hand for plupgrades between major versions.
 
If you install postgis, you have a postgis binary for each version of postgis for each version of postgres. After you connect to postgres database, you can switch the version of extension used, and several running simultaneous queries can be using different versions of postgis.

Packaging postgis+postgres in single package has a disadvantage - you can't use more than two extensions at the same time in that scheme.

How is it going to be handled?

Right, so maybe the solution would be to create a pg interface. Or again, we could use the content interface to create a way for extensions to be placed where PG can find them. This might need some coordinated work with Command Prompt to get the PG snap to look for extensions there. If you're interested in investing some time into it, so are we :) We are writing to upstreams also to know what they need (even though we can't promise it will be ready soon), but knowing what's lacking to snaps for it to be adopted certainly can modify our roadmap.

Again, if interested, we might be able to coordinate a meeting with other people involved to try to find a way :)

Best,

Julia

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: A snap for PostGis

Julia Palandri
Hi again, 

I was wondering if there are other questions I might be able to help with? Maybe somebody even started playing with snapcraft and has any doubts? Let me know in either case :)

Best,

On Fri, Dec 2, 2016 at 5:06 PM, Julia Palandri <[hidden email]> wrote:
Hi,

On Tue, Nov 29, 2016 at 1:10 PM Darafei "Komяpa" Praliaskouski <[hidden email]> wrote:

At the moment there are different snaps for each major version available (snap find postgres shows this), all of these in the stable channel. We are, however, working towards having smarter channels. Right now there is edge, which you can update with every commit to trunk. There's also beta, candidate, and of course stable.
We're aiming at having 9.6/stable, 9.6/candidate, 9.6/beta, 9.6/edge, 9.5/stable, 9.5/candidate, and so on, and also a "magic" latest family: latest/edge, latest/beta, etc. latest'stable would be the version you get when you ask the system to install any snap, without manually specifying the major version and flavour.

This describes install, but not upgrade process.


Well, rather, I was trying to explain that with different "series" for each major version, updates should be handled ok.
 
If you upgrade by just replacing binaries, you should keep in mind postgres 9.6 can't work with database created in postgres 9.4 - so, you need pg_upgrade (+ several manual steps) to perform the upgrade.


Right. So, I've been talking with the people that has been helping with the postgres snap, and they told that maybe the content interface might be able to give a hand for plupgrades between major versions.
 
If you install postgis, you have a postgis binary for each version of postgis for each version of postgres. After you connect to postgres database, you can switch the version of extension used, and several running simultaneous queries can be using different versions of postgis.

Packaging postgis+postgres in single package has a disadvantage - you can't use more than two extensions at the same time in that scheme.

How is it going to be handled?

Right, so maybe the solution would be to create a pg interface. Or again, we could use the content interface to create a way for extensions to be placed where PG can find them. This might need some coordinated work with Command Prompt to get the PG snap to look for extensions there. If you're interested in investing some time into it, so are we :) We are writing to upstreams also to know what they need (even though we can't promise it will be ready soon), but knowing what's lacking to snaps for it to be adopted certainly can modify our roadmap.

Again, if interested, we might be able to coordinate a meeting with other people involved to try to find a way :)

Best,

Julia



--

Julia

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel