[QGIS-Developer] OTB 6 and Processing

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[QGIS-Developer] OTB 6 and Processing

pcav
Hi all,
a new major version of OTB has been released. In the meantime, the OTB
provider has been removed from Processing on master.
Does anyone have a plan to keep the (very important IMHO) QGIS-OTB
integration working?
I'm available for help if necessary.
All the best, and thanks.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

pcav
Il 15/05/2017 10:13, Rashad Kanavath ha scritto:

> I had done some work in fixing some qgis otb stuff but was not
> integrated yet. These enhancements will reduce some code and improve otb
> processing.
>
> We @CS had contacted Victor olaya regarding this in Jaunary 2017. He
> had informed us about some major changes such as keeping all algorithms
> out of qgis processing plugin and some other stuff. since then there was
> not much communication and all those changes keep sitting on my backup
> disk like that..
>
> I would be able to help if anyone willing to integrate these changes
> back and adding otb processing plugin.
>
> regarding integration of QGIS-OTB, AFAIK, there is no plan or we simply
> don't know where to go.
>
> we are very much interested to hear back from qgis developer team.

Thanks Rashad for your prompt reply. I think Victor, Alex, and Nyall
(who is refactoring Processing for QGIS 3 and porting it to C++) are in
the best position to reply.
From my point of view, I think I'll interpret the feelings and needs of
thousands of users saying that we cannot miss OTB from Processing. Same
for R, this has been raised recently on the mailing list.
I think the steps are:
* create an independent provider (sub-plugin of Processing) for OTB
* publish it on the official repo.
Rashad, would you be willing to take ownership of that plugin?
All the best, and thanks again.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Julien Michel-2
Le 15/05/2017 à 10:21, Paolo Cavallini a écrit :
> Il 15/05/2017 10:13, Rashad Kanavath ha scritto:
>
> * create an independent provider (sub-plugin of Processing) for OTB
> * publish it on the official repo.
> Rashad, would you be willing to take ownership of that plugin?
I think OTB team as a whole would be glad to take ownership of that
plugin. Then we could add updates of the plugin tour release process and
ensure that it is always fresh and working. If this is ok four you, let
us know how to proceed.

Also, we have OTB users days 6-8 june @Toulouse, France, I think I
rembember that Victor lives not so far from Toulouse, so he is (as well
as anyone else who whants to join) most welcome to discuss this matter
with dev team and users.

Regards,

Julien

--
Julien MICHEL
CNES - DSO/SI/2A

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

pcav
Il 15/05/2017 10:24, Julien Michel ha scritto:

>> Rashad, would you be willing to take ownership of that plugin?
> I think OTB team as a whole would be glad to take ownership of that
> plugin. Then we could add updates of the plugin tour release process and
> ensure that it is always fresh and working. If this is ok four you, let
> us know how to proceed.

Great news, thanks.
AFAICT the first step is creating the subplugin, taking the removed code
from Processing. Probably Alex and/or Victor could give a hint on how to
best do this.

All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

volaya
Hi

The repo with the plugin already extracted and prepared is in here

https://github.com/volaya/qgis-otb-plugin

IT hasn't been published in the plugin repo, though. I think the OTB
team was ready to take ownership, so if anyone tells me the user name
or group that will accept it, i will transfer this.

The OTB user days sounds great, but unfortunately, I will not be at
home during those days (there is the spanish foss4g on 1-2 june, and i
will stay in Spain after that to visit my family). In any case, we can
set up a meeting and i can go to meet the OTB team in Toulouse in a
different date.

Let me know what I can do to move the repo ownership, so we can have
the OTB plugin released ASAP

About the R provider, we are in the same situation. The plugin is in
Alex's github account, but if anyone from the R community wants to
take ownership and take care of managing and releasing it, I guess we
can move it.

Cheers


2017-05-15 10:43 GMT+02:00 Paolo Cavallini <[hidden email]>:

> Il 15/05/2017 10:24, Julien Michel ha scritto:
>
>>> Rashad, would you be willing to take ownership of that plugin?
>> I think OTB team as a whole would be glad to take ownership of that
>> plugin. Then we could add updates of the plugin tour release process and
>> ensure that it is always fresh and working. If this is ok four you, let
>> us know how to proceed.
>
> Great news, thanks.
> AFAICT the first step is creating the subplugin, taking the removed code
> from Processing. Probably Alex and/or Victor could give a hint on how to
> best do this.
>
> All the best.
>
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
> https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

pcav
Il 15/05/2017 10:53, Victor Olaya ha scritto:
> Hi
>
> The repo with the plugin already extracted and prepared is in here
>
> https://github.com/volaya/qgis-otb-plugin

Great, thanks Victor for the prompt reply.
As said, ready to help with publication and testing.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Alexander Bruy
Hi,

I suggest not to rush with releasing and publishing any of the plugins.
Please don't forget that QGIS 2.14, 2.16 and 2.18 still have all providers
built in. For master I personally don't see any benefits in releasing
Processing providers right now. These plugins will be broken most of the
time as Processing suffers from major refactoring, QGIS API also is not
stablized yet.

As Victor said, if someone wants to take ownership on R provider, I wlll be
happy to move it. Otherwise I will try to do my best to keep it working.

Just my 2c.

2017-05-15 11:56 GMT+03:00 Paolo Cavallini <[hidden email]>:

> Il 15/05/2017 10:53, Victor Olaya ha scritto:
>> Hi
>>
>> The repo with the plugin already extracted and prepared is in here
>>
>> https://github.com/volaya/qgis-otb-plugin
>
> Great, thanks Victor for the prompt reply.
> As said, ready to help with publication and testing.
> All the best.
>
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
> https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer



--
Alexander Bruy
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

pcav
Hi Alex,

Il 15/05/2017 13:19, Alexander Bruy ha scritto:

> I suggest not to rush with releasing and publishing any of the plugins.
> Please don't forget that QGIS 2.14, 2.16 and 2.18 still have all providers
> built in. For master I personally don't see any benefits in releasing
> Processing providers right now. These plugins will be broken most of the
> time as Processing suffers from major refactoring, QGIS API also is not
> stablized yet.
>
> As Victor said, if someone wants to take ownership on R provider, I wlll be
> happy to move it. Otherwise I will try to do my best to keep it working.

thanks for this. I understand it may still be early for coding, unless
people @OTB have plans to support OTB6 also on 2.18 (if this is not too
complicated). I think it would however be good to start with the basics:
creating the repo/transferring owneship, etc.
In this way the work can start soon after the API have reasonably
stabilized (I imagine around June or July).
All the best.
--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

pcav
In reply to this post by pcav
Il 16/05/2017 10:08, Rashad Kanavath ha scritto:
> Hello Paolo,
>
> Is the processing plugin ported to c++? or just the core part with
> wrappers in python?

core in C++, with Python bindings. Nyall can be more precise on this.

> I am not clear on that. And this change will be for qgis3, users with
> qgis2.x need plugin the old way right?

right.
All the best.

--
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Alexander Bruy
Users of QGIS 2.x don't need separate plugin as OTB and all
other providers are part of Processing.

2017-05-16 11:16 GMT+03:00 Paolo Cavallini <[hidden email]>:

> Il 16/05/2017 10:08, Rashad Kanavath ha scritto:
>> Hello Paolo,
>>
>> Is the processing plugin ported to c++? or just the core part with
>> wrappers in python?
>
> core in C++, with Python bindings. Nyall can be more precise on this.
>
>> I am not clear on that. And this change will be for qgis3, users with
>> qgis2.x need plugin the old way right?
>
> right.
> All the best.
>
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
> https://www.google.com/trends/explore?date=all&geo=IT&q=qgis,arcgis
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer



--
Alexander Bruy
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Nyall Dawson
In reply to this post by pcav
On 16 May 2017 at 18:16, Paolo Cavallini <[hidden email]> wrote:
> Il 16/05/2017 10:08, Rashad Kanavath ha scritto:
>> Hello Paolo,
>>
>> Is the processing plugin ported to c++? or just the core part with
>> wrappers in python?
>
> core in C++, with Python bindings. Nyall can be more precise on this.

That's right. Gui will remain in python. Algs will mostly be python,
unless they benefit from the speed boost of the native c++ code. All
the algorithms which rely on external apps will remain Python.

If anyone's curious, here's a video demonstrating the current state of the work:
https://www.youtube.com/watch?v=6DIAc6ATOh0&feature=youtu.be

Not much to get excited about from an end user's perspective yet, but
what you see here is:
- a native buffer algorithm, fully written in c++
- background algorithm execution using task manager
- the new API which uses a dictionary of algorithm parameters
- a demo of dynamic property based parameters (will be exposed via the
UI just like data defined symbols/labels/etc)

Nyall
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Ricardo Filipe Soares Garcia da
It's looking great :)

Having processing algs run in the background will really improve the user experience IMO.

Having the possibility to use expressions in order to define parameters dynamically is really awesome too! In your video demo you effectively turned the fixed distance buffer algorithm into a variable distance buffer (maybe a hint that we'll need only one of those from now on?) ;)

Keep up the good work!


On Tue, May 16, 2017 at 9:41 AM, Nyall Dawson <[hidden email]> wrote:
On 16 May 2017 at 18:16, Paolo Cavallini <[hidden email]> wrote:
> Il 16/05/2017 10:08, Rashad Kanavath ha scritto:
>> Hello Paolo,
>>
>> Is the processing plugin ported to c++? or just the core part with
>> wrappers in python?
>
> core in C++, with Python bindings. Nyall can be more precise on this.

That's right. Gui will remain in python. Algs will mostly be python,
unless they benefit from the speed boost of the native c++ code. All
the algorithms which rely on external apps will remain Python.

If anyone's curious, here's a video demonstrating the current state of the work:
https://www.youtube.com/watch?v=6DIAc6ATOh0&feature=youtu.be

Not much to get excited about from an end user's perspective yet, but
what you see here is:
- a native buffer algorithm, fully written in c++
- background algorithm execution using task manager
- the new API which uses a dictionary of algorithm parameters
- a demo of dynamic property based parameters (will be exposed via the
UI just like data defined symbols/labels/etc)

Nyall
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer



--
___________________________ ___ __
Ricardo Garcia Silva

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Nyall Dawson
On 16 May 2017 at 19:40, Ricardo Filipe Soares Garcia da
<[hidden email]> wrote:

> It's looking great :)
>
> Having processing algs run in the background will really improve the user
> experience IMO.
>
> Having the possibility to use expressions in order to define parameters
> dynamically is really awesome too! In your video demo you effectively turned
> the fixed distance buffer algorithm into a variable distance buffer (maybe a
> hint that we'll need only one of those from now on?) ;)
>

That's the plan. Things won't scale well if we require separate
algorithms every time a parameter needs to be bound to a field or
expression (e.g. it's only a matter of time before someone needs a
"expression based buffer" alg), so the intention here is to allow
"data defined" parameters wherever it makes sense.

This has the potential to be very powerful. Imagine parameters within
models utilising expression variables, and some future "set variable"
processing algorithm....

Nyall
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

Alexander Bruy
In reply to this post by Alexander Bruy
Hi Rashad,

2017-05-16 11:46 GMT+03:00 Rashad Kanavath <[hidden email]>:
> okay.  current version needs update of lot of xml files, change in version
> number, list of application inside the otb plugin sources. I want to avoid
> that. And I think both OTB and QGIS developers will agree on this.
>
> instead of xml files, otb can directly output those csv files which are read
> by processing core.

I'm not against this, but IMHO such work should go for 3.0, as it requires
many changes in OTB provider. Pushing this changes in 2.18 is dangerous
(we already have broken SAGA support because of too fast switch to
SAGA 2.3.1) and may break OTB support in LTR version.

I think OTB team should take ownership on OTB provider plugin (it is hosted
in the Victor's repository now) and start impelement new functionality. For
QGIS 2.18 better to leave OTB support as is, or maybe backport some
fixes and description files to have OTB 5.4. and 5.6.

--
Alexander Bruy
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

volaya
I agree 100% on what Alex said.

Rashad, instead of adding your user, can I completely move the
ownership to you or to another account that is managed by the OTB
team?

2017-05-17 11:18 GMT+02:00 Alexander Bruy <[hidden email]>:

> Hi Rashad,
>
> 2017-05-16 11:46 GMT+03:00 Rashad Kanavath <[hidden email]>:
>> okay.  current version needs update of lot of xml files, change in version
>> number, list of application inside the otb plugin sources. I want to avoid
>> that. And I think both OTB and QGIS developers will agree on this.
>>
>> instead of xml files, otb can directly output those csv files which are read
>> by processing core.
>
> I'm not against this, but IMHO such work should go for 3.0, as it requires
> many changes in OTB provider. Pushing this changes in 2.18 is dangerous
> (we already have broken SAGA support because of too fast switch to
> SAGA 2.3.1) and may break OTB support in LTR version.
>
> I think OTB team should take ownership on OTB provider plugin (it is hosted
> in the Victor's repository now) and start impelement new functionality. For
> QGIS 2.18 better to leave OTB support as is, or maybe backport some
> fixes and description files to have OTB 5.4. and 5.6.
>
> --
> Alexander Bruy
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OTB 6 and Processing

volaya
To transfer the repo i need permission to create new repos under that
org. If you can give me rights to do it, I will do the transfer and
then you can revoke them

Thanks

2017-05-22 10:22 GMT+02:00 Rashad Kanavath <[hidden email]>:

>
>
> You can transfer to orfeotoolbox. this is the official github account and
> all devs have access to it.
>
> https://github.com/orfeotoolbox
>
>
>
> On Wed, May 17, 2017 at 12:04 PM, Victor Olaya <[hidden email]> wrote:
>>
>> I agree 100% on what Alex said.
>>
>> Rashad, instead of adding your user, can I completely move the
>> ownership to you or to another account that is managed by the OTB
>> team?
>>
>> 2017-05-17 11:18 GMT+02:00 Alexander Bruy <[hidden email]>:
>> > Hi Rashad,
>> >
>> > 2017-05-16 11:46 GMT+03:00 Rashad Kanavath <[hidden email]>:
>> >> okay.  current version needs update of lot of xml files, change in
>> >> version
>> >> number, list of application inside the otb plugin sources. I want to
>> >> avoid
>> >> that. And I think both OTB and QGIS developers will agree on this.
>> >>
>> >> instead of xml files, otb can directly output those csv files which are
>> >> read
>> >> by processing core.
>> >
>> > I'm not against this, but IMHO such work should go for 3.0, as it
>> > requires
>> > many changes in OTB provider. Pushing this changes in 2.18 is dangerous
>> > (we already have broken SAGA support because of too fast switch to
>> > SAGA 2.3.1) and may break OTB support in LTR version.
>> >
>> > I think OTB team should take ownership on OTB provider plugin (it is
>> > hosted
>> > in the Victor's repository now) and start impelement new functionality.
>> > For
>> > QGIS 2.18 better to leave OTB support as is, or maybe backport some
>> > fixes and description files to have OTB 5.4. and 5.6.
>> >
>> > --
>> > Alexander Bruy
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > [hidden email]
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
>
> --
> Regards,
>    Rashad
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer