Re: refactor 'run.pl' to a python class definition ahead of Open MPI integration

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: refactor 'run.pl' to a python class definition ahead of Open MPI integration

Daryl Van Dyke

Hi -

Long time listener, first time caller...

I have been thinking about a python class that leverage OpenDroneMap since the first time I looked at the perl driver.  I don't know perl, but could discern the general pattern.

I'd like to start a pseudocode class definition diagram  for what it might look like.  I see it as an object of -model- which would be initialized from an initial set of images.  The sequential steps of the perl script would become methods, with the ability to keep track of which results are associated with (result from) which parameter configs.

If I wanted to take a stab at this - could I open a wiki page on GitHub?  I don't want to appear dis-respectful as a newcomer, but perl is a barrier for me and I see many benefits to transitioning the driver to python 2.7.

Daryl

On May 8, 2015 8:34 PM, "Stephen Mather" <[hidden email]> wrote:
Shared, or the messy part of passing data around...

Looking at the portions of the toolchain:

resize -- easily can be massively parallelized (also not a bottleneck)
keypoints -- same, but is a slow process now
match -- harder to parallelize without a priori knowledge of likely matches
bundler -- currently not fully parallelized and a major bottleneck
cmvs/pmvs -- CMVS chops the scene into bite-sized chunks, making PMVS work within smaller, reassembleable scene chunks. Ideal for distribution
meshing -- not sure, but fast to run relative to rest of toolchain
texturing -- not sure, but fast to run relative to rest of toolchain
georeferencing -- not sure, but fast to run relative to rest of toolchain
orthophoto -- not sure, but fast to run relative to rest of toolchain

match is probably the hardest part to efficiently parallize without shared storage, but with some a priori knowledge of likely matching images (e.g. exif geolocation), something clever might be done to overcome this limitation.

Cheers,
Best,
Steve




On Mon, Apr 27, 2015 at 6:38 PM, Alex Mandel <[hidden email]> wrote:
On 04/27/2015 09:18 AM, Bales, Donald J. wrote:
> All,
>
> Forgive me if you feel this inquiry is misplaced, but I was wondering if OpenDroneMap has an OpenMPI implementation, or if you know of any OpenMPI implementation for image processing?  If not, is there any interest in the OpenDroneMap community to create an OpenMPI implementation so one can leverage high performance computing platforms?
> Sincerely,
>
> Donald J. Bales
> <a href="tel:%28630%29%20776-0071" value="+16307760071" target="_blank">(630) 776-0071
>

We have been talking about converting the current Perl master script to
something in Python (possibly flask based). At that point there would be
good places to hook in an MPI library for python.

The tricky part is going to be an efficient means of distributing the
data. If you have a compute cluster this shouldn't be an issue (assuming
you have some sort of shared storage). If you use Amazon it might mean
you have all your nodes reads from the same S3, etc.

Thanks,
Alex

_______________________________________________
OpenDroneMap-dev mailing list
[hidden email]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/opendronemap-dev


_______________________________________________
OpenDroneMap-dev mailing list
[hidden email]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/opendronemap-dev


_______________________________________________
OpenDroneMap-dev mailing list
[hidden email]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/opendronemap-dev
Reply | Threaded
Open this post in threaded view
|

Re: refactor 'run.pl' to a python class definition ahead of Open MPI integration

Stephen Mather-2
Hi Daryl,

Wiki's and pull requests always welcome.  Jump right in.
This would address issue 20, which I consider to be top priority to ensure that the widest number of people can participate:

https://github.com/OpenDroneMap/OpenDroneMap/issues/20

Cheers,
Best,
Steve




On Fri, May 8, 2015 at 10:04 PM, Daryl Van Dyke <[hidden email]> wrote:

Hi -

Long time listener, first time caller...

I have been thinking about a python class that leverage OpenDroneMap since the first time I looked at the perl driver.  I don't know perl, but could discern the general pattern.

I'd like to start a pseudocode class definition diagram  for what it might look like.  I see it as an object of -model- which would be initialized from an initial set of images.  The sequential steps of the perl script would become methods, with the ability to keep track of which results are associated with (result from) which parameter configs.

If I wanted to take a stab at this - could I open a wiki page on GitHub?  I don't want to appear dis-respectful as a newcomer, but perl is a barrier for me and I see many benefits to transitioning the driver to python 2.7.

Daryl

On May 8, 2015 8:34 PM, "Stephen Mather" <[hidden email]> wrote:
Shared, or the messy part of passing data around...

Looking at the portions of the toolchain:

resize -- easily can be massively parallelized (also not a bottleneck)
keypoints -- same, but is a slow process now
match -- harder to parallelize without a priori knowledge of likely matches
bundler -- currently not fully parallelized and a major bottleneck
cmvs/pmvs -- CMVS chops the scene into bite-sized chunks, making PMVS work within smaller, reassembleable scene chunks. Ideal for distribution
meshing -- not sure, but fast to run relative to rest of toolchain
texturing -- not sure, but fast to run relative to rest of toolchain
georeferencing -- not sure, but fast to run relative to rest of toolchain
orthophoto -- not sure, but fast to run relative to rest of toolchain

match is probably the hardest part to efficiently parallize without shared storage, but with some a priori knowledge of likely matching images (e.g. exif geolocation), something clever might be done to overcome this limitation.

Cheers,
Best,
Steve




On Mon, Apr 27, 2015 at 6:38 PM, Alex Mandel <[hidden email]> wrote:
On 04/27/2015 09:18 AM, Bales, Donald J. wrote:
> All,
>
> Forgive me if you feel this inquiry is misplaced, but I was wondering if OpenDroneMap has an OpenMPI implementation, or if you know of any OpenMPI implementation for image processing?  If not, is there any interest in the OpenDroneMap community to create an OpenMPI implementation so one can leverage high performance computing platforms?
> Sincerely,
>
> Donald J. Bales
> <a href="tel:%28630%29%20776-0071" value="+16307760071" target="_blank">(630) 776-0071
>

We have been talking about converting the current Perl master script to
something in Python (possibly flask based). At that point there would be
good places to hook in an MPI library for python.

The tricky part is going to be an efficient means of distributing the
data. If you have a compute cluster this shouldn't be an issue (assuming
you have some sort of shared storage). If you use Amazon it might mean
you have all your nodes reads from the same S3, etc.

Thanks,
Alex

_______________________________________________
OpenDroneMap-dev mailing list
[hidden email]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/opendronemap-dev


_______________________________________________
OpenDroneMap-dev mailing list
[hidden email]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/opendronemap-dev



_______________________________________________
OpenDroneMap-dev mailing list
[hidden email]
http://lists.osgeo.org/cgi-bin/mailman/listinfo/opendronemap-dev