Squeezing the most out of ossim

classic Classic list List threaded Threaded
64 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Squeezing the most out of ossim

Andyjo
So i finally have a working dev copy, The obvious the next progression is to get some better performance out of the applications i'm using (mainly just orthoigen for now).
I've done a little bench marking and poking around, and can't seem to figure out what the limiting factor is when orthorectifying an image. When i hand orthoigen a list of 50 or so 177mb tiffs, and let it go, it only uses about 13% cpu, and a bunch of memory, i checked out the file i/o and it doesn't look like that's bottle-necking at all (all data is on SSDs). So i moved on to trying to get MPI working... got it working, but the results were for of weird, image sizes seemed to be scaled by the number of cpus i threw at it... anyone ever see this issue? It also didn't seem to help very much with speed.
Some other settings i've been playing with were the cache_size, and ossim_threads, the cache_size seems to help a little bit, in particulate when dealing with more files.
So how to you guys get ossim to use every bit of processing power the computer has? Process monitor seems to tell me there's a lot of File I/O... maybe RAM disks?!?
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

David Burken
Hi,

With 70 inputs I suspect you have I/O limitations.  We don't have hardly
any multithreaded stuff under the hood.  The resampler gobbles a lot of
CPU.  The mpi stuff is tricky and if your on the same machine you're
sharing the memory resources so I'm not sure it would be worth it.  On
the wish list is a thread safe getTile.  That would be awesome.  Oscar
Kramer has been experimenting with multithreaded image chains which is
another approach.  As far as tuning some keywords:

cache_size: 2048
// cache_size: For GUI interaction this can make a big difference. For
command line stuff you only need a small cache on the left hand side of
the resampler.

// ossim_threads:  Currently only used in a couple of places,
ossimImageUtil, and ossimImageElevationDatabase.  The ossim-preproc app
uses ossimImageUtil and can thread the preprocessing of images, i.e.
overviews, histograms on a per image bases. ossimImageElevationDatabase
is used at start up if you use an image_directory in your prefs like:
elevation_manager.elevation_source1.type: image_directory
ossim_threads: 4

// ---
// Kakadu threads:
// ---
kakadu_threads: 4

Only good for ossim kakadu plugin.  Definitely affects speed!

The resampler filter is also a speed issue.  Controlled via command line
option.  Using nearest neighbor is good if there's no
rotation/reprojection and you can get by with it.

I know not much help...
Dave


On 02/22/2013 04:12 PM, Andyjo wrote:

> So i finally have a working dev copy, The obvious the next progression is to
> get some better performance out of the applications i'm using (mainly just
> orthoigen for now).
> I've done a little bench marking and poking around, and can't seem to figure
> out what the limiting factor is when orthorectifying an image. When i hand
> orthoigen a list of 50 or so 177mb tiffs, and let it go, it only uses about
> 13% cpu, and a bunch of memory, i checked out the file i/o and it doesn't
> look like that's bottle-necking at all (all data is on SSDs). So i moved on
> to trying to get MPI working... got it working, but the results were for of
> weird, image sizes seemed to be scaled by the number of cpus i threw at
> it... anyone ever see this issue? It also didn't seem to help very much with
> speed.
> Some other settings i've been playing with were the cache_size, and
> ossim_threads, the cache_size seems to help a little bit, in particulate
> when dealing with more files.
> So how to you guys get ossim to use every bit of processing power the
> computer has? Process monitor seems to tell me there's a lot of File I/O...
> maybe RAM disks?!?
>
>
>
>
> --
> View this message in context: http://osgeo-org.1560.n6.nabble.com/Squeezing-the-most-out-of-ossim-tp5036271.html
> Sent from the Ossim-developer mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

oscarkramer
I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

oscarkramer
My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Andyjo
Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Peter Borissow
What images are you ortho'ing?

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter





From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim

Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer



------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Andyjo

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.

I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.

 

 

 

From: Peter Borissow [mailto:[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

What images are you ortho'ing?

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter

 

 


From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim


Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

oscarkramer
In reply to this post by Andyjo
I don't have a special orthoigen. It is osgeo trunk. Starting at line 333 of ossim/src/ossim/parallel/ossimOrthoIgen.cpp are all the options for debugging MT.

I would follow Peter's advice and ortho individual images on separate nodes first. Render them to a common GSD ("--meters" option) if feasible. Then you mosaic. Chop up if necessary.
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Peter Borissow
In reply to this post by Andyjo
Unfortunately, ossim doesn't mosiac thousands of images very well.

But...

If the images are tiled, you can mosaic tens of thousands of images using a .rpf file.

A couple years ago, we developed an ill-named ".rpf" file format. It's basically a text file with a list of images you want to mosaic. ossim will handle the images as one giant tiled image. We used this to mosaic USDA NAIP data, GDEM, SRTM, DTED, RPF, etc. Basically, anything that is tiled.

I've attached a sample ".rpf" file. The first line represents the MBR of all the images, and the number of bands. After that, you'll see a list of paths to individual "tiles", along with their corner coordinates.

Couple constraints:
 - Images must be in the same gsd
 - Images must be in the same projection
 - Images must be in the same radiometry



So one approach would be to ortho all your buckeye data individually, chop them up into even tiles, and mosaic them using the .rpf handler.


Peter






From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [mailto:[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
What images are you ortho'ing?

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter

 
 

From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim

Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer




------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer

GNCJNCS.rpf (108K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Norman Vine
In reply to this post by Andyjo
<base href="x-msg://1061/">

This sounds like a process begging for something like OMAR

e.g. ortho'ing individual images then virtual mosaicing on demand



On Feb 25, 2013, at 12:16 PM, "Andrew D. Johnson" <[hidden email]> wrote:

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [mailto:peter.borissow@yahoo.com] 
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 

What images are you ortho'ing? 

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter

 
 

From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]> 
Cc: "[hidden email]" <[hidden email]> 
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim


Oscar, 
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]] 
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Andyjo
In reply to this post by Peter Borissow

Thanks peter, I’ll look into tiling up the images after they’re ortho’d… thinking about it a little more, how does ossim handle overlapping tiles, wouldn’t the combiner then just see it as one image?

Also, is there any sort of limitation to what file type is contained in the .rpf, anything ossim can handle?

 

From: Peter Borissow [mailto:[hidden email]]
Sent: Monday, February 25, 2013 12:29 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

Unfortunately, ossim doesn't mosiac thousands of images very well.

But...

If the images are tiled, you can mosaic tens of thousands of images using a .rpf file.

A couple years ago, we developed an ill-named ".rpf" file format. It's basically a text file with a list of images you want to mosaic. ossim will handle the images as one giant tiled image. We used this to mosaic USDA NAIP data, GDEM, SRTM, DTED, RPF, etc. Basically, anything that is tiled.

I've attached a sample ".rpf" file. The first line represents the MBR of all the images, and the number of bands. After that, you'll see a list of paths to individual "tiles", along with their corner coordinates.

Couple constraints:
 - Images must be in the same gsd
 - Images must be in the same projection
 - Images must be in the same radiometry



So one approach would be to ortho all your buckeye data individually, chop them up into even tiles, and mosaic them using the .rpf handler.


Peter


 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

 

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.

I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.

 

 

 

From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

What images are you ortho'ing?

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter

 

 


From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim


Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer

 


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

Peter Borissow
You can specify the "z-order" when combining images programatically or using orthoigen.

Note that you can't have overlapping images in the .rpf file. The tiles must line up exactly.

And yes, the .rpf can handle any image format.

Peter



From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:47 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

Thanks peter, I’ll look into tiling up the images after they’re ortho’d… thinking about it a little more, how does ossim handle overlapping tiles, wouldn’t the combiner then just see it as one image?
Also, is there any sort of limitation to what file type is contained in the .rpf, anything ossim can handle?
 
From: Peter Borissow [mailto:[hidden email]]
Sent: Monday, February 25, 2013 12:29 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
Unfortunately, ossim doesn't mosiac thousands of images very well.

But...

If the images are tiled, you can mosaic tens of thousands of images using a .rpf file.

A couple years ago, we developed an ill-named ".rpf" file format. It's basically a text file with a list of images you want to mosaic. ossim will handle the images as one giant tiled image. We used this to mosaic USDA NAIP data, GDEM, SRTM, DTED, RPF, etc. Basically, anything that is tiled.

I've attached a sample ".rpf" file. The first line represents the MBR of all the images, and the number of bands. After that, you'll see a list of paths to individual "tiles", along with their corner coordinates.

Couple constraints:
 - Images must be in the same gsd
 - Images must be in the same projection
 - Images must be in the same radiometry



So one approach would be to ortho all your buckeye data individually, chop them up into even tiles, and mosaic them using the .rpf handler.


Peter


 
 

From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim
 
Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
What images are you ortho'ing?

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter
 
 

From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim

Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
 



------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Squeezing the most out of ossim

jono
In reply to this post by Andyjo
Andrew-

One strategy we've used in the past with DSS and 100's to 1-2000 images:

1) Calculate the image footprints (a priori from image events data)
2) Generate tile scheme for block
3) Intersect footprints and tiles to create a list of tiles and the images they contain
4) Using multithreaded code, pass the list to multiple orthoigen instances with the closest to center combiner selected.

Lets you go straight to mosaic bypassing the intermediate ortho.

For complex terrain, you may want to build pyramids into the images and ortho out a low res image at a couple meter resolution and extract the footprints.

Jon


On Mon, Feb 25, 2013 at 12:47 PM, Andrew D. Johnson <[hidden email]> wrote:

Thanks peter, I’ll look into tiling up the images after they’re ortho’d… thinking about it a little more, how does ossim handle overlapping tiles, wouldn’t the combiner then just see it as one image?

Also, is there any sort of limitation to what file type is contained in the .rpf, anything ossim can handle?

 

From: Peter Borissow [mailto:[hidden email]]
Sent: Monday, February 25, 2013 12:29 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

Unfortunately, ossim doesn't mosiac thousands of images very well.

But...

If the images are tiled, you can mosaic tens of thousands of images using a .rpf file.

A couple years ago, we developed an ill-named ".rpf" file format. It's basically a text file with a list of images you want to mosaic. ossim will handle the images as one giant tiled image. We used this to mosaic USDA NAIP data, GDEM, SRTM, DTED, RPF, etc. Basically, anything that is tiled.

I've attached a sample ".rpf" file. The first line represents the MBR of all the images, and the number of bands. After that, you'll see a list of paths to individual "tiles", along with their corner coordinates.

Couple constraints:
 - Images must be in the same gsd
 - Images must be in the same projection
 - Images must be in the same radiometry



So one approach would be to ortho all your buckeye data individually, chop them up into even tiles, and mosaic them using the .rpf handler.


Peter


 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

 

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.

I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.

 

 

 

From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

What images are you ortho'ing?

You might be better off to ortho individual images and mosaic them together.

Better yet, chop up the orthos into tiles and use the .rpf file handler.

Peter

 

 


From: Andrew D. Johnson <[hidden email]>
To: Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 11:54 AM
Subject: Re: [OSSIM] Squeezing the most out of ossim


Oscar,
Do you have a 'special' version of orthoigen? I grepped through the entire source and didn't find anything that resembles '-*mt*' besides some GSD flags.
I tried playing around with the --threads flag, and I got some really odd results, lots of null, and polka-dot style imagery... I think the small tiles are in the correct place, just... not filling in the entire image.
I guess another question I have is, is it better to orthorectify all my imagery first, then mosaic it together? Or do it all in one huge process? I have a pseudo cluster I could dish out orthoigen jobs to, and then I guess I could run orthoigen --disable-elev with a combiner type? Just throwing out ideas!
I've been looking into how the sequencer uses the theThreadCount field to see if I can figure out why I have spotty imagery in the end...

-----Original Message-----
From: Oscar L. Kramer [mailto:[hidden email]]
Sent: Monday, February 25, 2013 10:49 AM
To: David Burken; Andrew D. Johnson
Cc: [hidden email]
Subject: RE: [OSSIM] Squeezing the most out of ossim

My suggestion is to look at the ossimCacheTileSource and related classes for a way to implement the cache in a non-centralized thread-friendly way. Do you have a good thread debugging tool? It will be very difficult otherwise. Look at ossimMtDebug also where you can switch on different code (via orthoigen's command line "--mt-*" set of options).

________________________________________
From: Oscar L. Kramer [[hidden email]]
Sent: Monday, February 25, 2013 10:29
To: David Burken; Andyjo
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

I implemented multithreading in orthoigen using OpenThreads. It is experimental so requires the "--threads" option on the command line. For experimental purposes, you can specify the number of threads to launch with "--threads N". If you leave off N it will default to some multiple (I think 1.5) of the number of cores.

I had limited success during my trials. I was able to get about a 2X speedup, then hit a wall. It seemed to be because the cache is centralized and all threads have to wait. IO was a big issue too.

To get around the IO, I implemented a multithreaded image handler that works two ways (see the source at ossim/parallel/ossimImageHandlerMtAdaptor.h|cpp to see how to switch between the two approaches). There is an "adaptor" for image chain as well. I was experimenting with one image handler shared by all threads, and one image handler per thread. But because of the cache bottleneck, I could not determine which approach was faster. Take a look at all the multithread (MT) code, mostly in the ossim/parallel subdir.

I also disabled the tile cache in the chain, but found that the cache in the image handler was using the same centralized control so no advantage was gained.

Good luck. Let us know what you find. I may revisit that code one day...

Oscar
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer

 


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer



------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Buckeye Sensor Model

Peter Borissow
In reply to this post by Peter Borissow
Andrew-
    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.

The more people that can use Buckeye the better.

Peter




From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [mailto:[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
What images are you ortho'ing?

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Buckeye Sensor Model

Andyjo

Peter, of course!

I’m a little hesitant right now, because I think I’m trashing the stack somewhere in it…. I can ortho out to tifs fine, but for some reason when I try to use any plugin to write output, the application crashes at the return from write(). We’re also doing some extensive lens modeling here and I hope to get the correct distortion model hooked in as well – honestly I pretty much just copied the Applanix model as a start and went from there with our metadata format – still need to do the due-diligence and double check all of the math for my own sanity, but I am getting images projected to within 0.5-2m of our other processing methods.

 

-Andy

 

From: Peter Borissow [mailto:[hidden email]]
Sent: Tuesday, February 26, 2013 9:13 AM
To: Andrew D. Johnson
Cc: [hidden email]
Subject: Buckeye Sensor Model

 

Andrew-

    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.

 

The more people that can use Buckeye the better.

 

Peter

 

 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

 

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.

I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.

 

 

 

From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

What images are you ortho'ing?


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Buckeye Sensor Model

Peter Borissow
That's Great!

I haven't been around Buckeye in a couple years but as I recall they were planning to collect LIDAR inline with the images. It would be really nice if a user could specify an elevation source to use when ortho'ing the images with orthoigen (vs loading all the LIDAR into the elevation manager). I believe all the hooks are in there now and all that's needed is to update the orthoigen app. Just a thought...

BTW, Oscar Kramer (cc'd) is a great resource for any sensor modelling issues you may have.

Best of luck!
Peter



From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Tuesday, February 26, 2013 9:20 AM
Subject: RE: Buckeye Sensor Model

Peter, of course!
I’m a little hesitant right now, because I think I’m trashing the stack somewhere in it…. I can ortho out to tifs fine, but for some reason when I try to use any plugin to write output, the application crashes at the return from write(). We’re also doing some extensive lens modeling here and I hope to get the correct distortion model hooked in as well – honestly I pretty much just copied the Applanix model as a start and went from there with our metadata format – still need to do the due-diligence and double check all of the math for my own sanity, but I am getting images projected to within 0.5-2m of our other processing methods.
 
-Andy
 
From: Peter Borissow [mailto:[hidden email]]
Sent: Tuesday, February 26, 2013 9:13 AM
To: Andrew D. Johnson
Cc: [hidden email]
Subject: Buckeye Sensor Model
 
Andrew-
    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.
 
The more people that can use Buckeye the better.
 
Peter
 
 
 

From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim
 
Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
What images are you ortho'ing?



------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Buckeye Sensor Model

Andyjo

Thanks Peter,

And yes, we do collect LIDAR inline, I was actually wondering if there was a way to do that rather than updating the elevation manager every time we collect.

Would it be best to update my metadata converter (*.geom) to include the path to a lidar dataset, or would that be more of a orthoigen input?

 

I also have some updates for ossim-mosaic which I’ll drop a ticket for with attached source, just updated it to include all of the combiners and mosaic types.

 

-Andy

 

 

From: Peter Borissow [mailto:[hidden email]]
Sent: Tuesday, February 26, 2013 9:36 AM
To: Andrew D. Johnson
Cc: [hidden email]; Oscar Kramer
Subject: Re: Buckeye Sensor Model

 

That's Great!

I haven't been around Buckeye in a couple years but as I recall they were planning to collect LIDAR inline with the images. It would be really nice if a user could specify an elevation source to use when ortho'ing the images with orthoigen (vs loading all the LIDAR into the elevation manager). I believe all the hooks are in there now and all that's needed is to update the orthoigen app. Just a thought...

BTW, Oscar Kramer (cc'd) is a great resource for any sensor modelling issues you may have.

Best of luck!
Peter

 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Tuesday, February 26, 2013 9:20 AM
Subject: RE: Buckeye Sensor Model

 

Peter, of course!

I’m a little hesitant right now, because I think I’m trashing the stack somewhere in it…. I can ortho out to tifs fine, but for some reason when I try to use any plugin to write output, the application crashes at the return from write(). We’re also doing some extensive lens modeling here and I hope to get the correct distortion model hooked in as well – honestly I pretty much just copied the Applanix model as a start and went from there with our metadata format – still need to do the due-diligence and double check all of the math for my own sanity, but I am getting images projected to within 0.5-2m of our other processing methods.

 

-Andy

 

From: Peter Borissow [[hidden email]]
Sent: Tuesday, February 26, 2013 9:13 AM
To: Andrew D. Johnson
Cc: [hidden email]
Subject: Buckeye Sensor Model

 

Andrew-

    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.

 

The more people that can use Buckeye the better.

 

Peter

 

 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

 

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.

I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.

 

 

 

From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

What images are you ortho'ing?

 


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Buckeye Sensor Model

Garrett Potts-2
<base href="x-msg://1298/">Hello Guys:

I have written a buckey sensor in OSSIM and can be found under:

include/ossim/projection/ossimBuckeyeSensor.h
src/ossim/projection/ossimBuckeyeSensor.h


If we are missing anything or need to add any thing please let me know.  You should be able to use this as a base?  


Take care

Garrett


On Feb 26, 2013, at 10:07 AM, Andrew D. Johnson <[hidden email]> wrote:

Thanks Peter,
And yes, we do collect LIDAR inline, I was actually wondering if there was a way to do that rather than updating the elevation manager every time we collect.
Would it be best to update my metadata converter (*.geom) to include the path to a lidar dataset, or would that be more of a orthoigen input?
 
I also have some updates for ossim-mosaic which I’ll drop a ticket for with attached source, just updated it to include all of the combiners and mosaic types.
 
-Andy
 
 
From: Peter Borissow [mailto:peter.borissow@yahoo.com] 
Sent: Tuesday, February 26, 2013 9:36 AM
To: Andrew D. Johnson
Cc: [hidden email]; Oscar Kramer
Subject: Re: Buckeye Sensor Model
 
That's Great! 

I haven't been around Buckeye in a couple years but as I recall they were planning to collect LIDAR inline with the images. It would be really nice if a user could specify an elevation source to use when ortho'ing the images with orthoigen (vs loading all the LIDAR into the elevation manager). I believe all the hooks are in there now and all that's needed is to update the orthoigen app. Just a thought...

BTW, Oscar Kramer (cc'd) is a great resource for any sensor modelling issues you may have. 

Best of luck!
Peter
 
 

From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]> 
Cc: "[hidden email]" <[hidden email]> 
Sent: Tuesday, February 26, 2013 9:20 AM
Subject: RE: Buckeye Sensor Model
 
Peter, of course!
I’m a little hesitant right now, because I think I’m trashing the stack somewhere in it…. I can ortho out to tifs fine, but for some reason when I try to use any plugin to write output, the application crashes at the return from write(). We’re also doing some extensive lens modeling here and I hope to get the correct distortion model hooked in as well – honestly I pretty much just copied the Applanix model as a start and went from there with our metadata format – still need to do the due-diligence and double check all of the math for my own sanity, but I am getting images projected to within 0.5-2m of our other processing methods.
 
-Andy
 
From: Peter Borissow [[hidden email]] 
Sent: Tuesday, February 26, 2013 9:13 AM
To: Andrew D. Johnson
Cc: [hidden email]
Subject: Buckeye Sensor Model
 
Andrew-
    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.
 
The more people that can use Buckeye the better.
 
Peter
 
 
 

From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]> 
Cc: "[hidden email]" <[hidden email]> 
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim
 
Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [[hidden email]] 
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
What images are you ortho'ing?

 

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Buckeye Sensor Model

Peter Borissow
Cool!



From: Garrett Potts <[hidden email]>
To: Andrew D. Johnson <[hidden email]>
Cc: Peter Borissow <[hidden email]>; "[hidden email]" <[hidden email]>
Sent: Tuesday, February 26, 2013 10:18 AM
Subject: Re: [OSSIM] Buckeye Sensor Model

Hello Guys:

I have written a buckey sensor in OSSIM and can be found under:

include/ossim/projection/ossimBuckeyeSensor.h
src/ossim/projection/ossimBuckeyeSensor.h


If we are missing anything or need to add any thing please let me know.  You should be able to use this as a base?  


Take care

Garrett


On Feb 26, 2013, at 10:07 AM, Andrew D. Johnson <[hidden email]> wrote:

Thanks Peter,
And yes, we do collect LIDAR inline, I was actually wondering if there was a way to do that rather than updating the elevation manager every time we collect.
Would it be best to update my metadata converter (*.geom) to include the path to a lidar dataset, or would that be more of a orthoigen input?
 
I also have some updates for ossim-mosaic which I’ll drop a ticket for with attached source, just updated it to include all of the combiners and mosaic types.
 
-Andy
 
 
From: Peter Borissow [mailto:peter.borissow@yahoo.com] 
Sent: Tuesday, February 26, 2013 9:36 AM
To: Andrew D. Johnson
Cc: [hidden email]; Oscar Kramer
Subject: Re: Buckeye Sensor Model
 
That's Great! 

I haven't been around Buckeye in a couple years but as I recall they were planning to collect LIDAR inline with the images. It would be really nice if a user could specify an elevation source to use when ortho'ing the images with orthoigen (vs loading all the LIDAR into the elevation manager). I believe all the hooks are in there now and all that's needed is to update the orthoigen app. Just a thought...

BTW, Oscar Kramer (cc'd) is a great resource for any sensor modelling issues you may have. 

Best of luck!
Peter
 
 

From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]> 
Cc: "[hidden email]" <[hidden email]> 
Sent: Tuesday, February 26, 2013 9:20 AM
Subject: RE: Buckeye Sensor Model
 
Peter, of course!
I’m a little hesitant right now, because I think I’m trashing the stack somewhere in it…. I can ortho out to tifs fine, but for some reason when I try to use any plugin to write output, the application crashes at the return from write(). We’re also doing some extensive lens modeling here and I hope to get the correct distortion model hooked in as well – honestly I pretty much just copied the Applanix model as a start and went from there with our metadata format – still need to do the due-diligence and double check all of the math for my own sanity, but I am getting images projected to within 0.5-2m of our other processing methods.
 
-Andy
 
From: Peter Borissow [[hidden email]] 
Sent: Tuesday, February 26, 2013 9:13 AM
To: Andrew D. Johnson
Cc: [hidden email]
Subject: Buckeye Sensor Model
 
Andrew-
    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.
 
The more people that can use Buckeye the better.
 
Peter
 
 
 

From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]> 
Cc: "[hidden email]" <[hidden email]> 
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim
 
Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.
I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.
 
 
 
From: Peter Borissow [[hidden email]] 
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim
 
What images are you ortho'ing?
 
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer




------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Buckeye Sensor Model

oscarkramer
In reply to this post by Andyjo
Andrew,

Unfortunately the .geom file is not the place to list LIDAR elevation sources. That file is read as part of the getImageGeometry() scheme in the image handlers and wouldn't be seen by the elevation manager (as currently implemented). Use the "-K <prefs_kwd:value>" option on orthoigen to list additional elevation paths, or create a dedicated preferences file with the relevant elevation files for each specific dataset. I wis we had a way of nesting preferences, so that a dataset-specific preference file could reference a master defaults preferences file.

Oscar

From: Andrew D. Johnson [[hidden email]]
Sent: Tuesday, February 26, 2013 10:07
To: Peter Borissow
Cc: [hidden email]; Oscar L. Kramer
Subject: RE: Buckeye Sensor Model

Thanks Peter,

And yes, we do collect LIDAR inline, I was actually wondering if there was a way to do that rather than updating the elevation manager every time we collect.

Would it be best to update my metadata converter (*.geom) to include the path to a lidar dataset, or would that be more of a orthoigen input?

 

I also have some updates for ossim-mosaic which I’ll drop a ticket for with attached source, just updated it to include all of the combiners and mosaic types.

 

-Andy

 

 

From: Peter Borissow [mailto:[hidden email]]
Sent: Tuesday, February 26, 2013 9:36 AM
To: Andrew D. Johnson
Cc: [hidden email]; Oscar Kramer
Subject: Re: Buckeye Sensor Model

 

That's Great!

I haven't been around Buckeye in a couple years but as I recall they were planning to collect LIDAR inline with the images. It would be really nice if a user could specify an elevation source to use when ortho'ing the images with orthoigen (vs loading all the LIDAR into the elevation manager). I believe all the hooks are in there now and all that's needed is to update the orthoigen app. Just a thought...

BTW, Oscar Kramer (cc'd) is a great resource for any sensor modelling issues you may have.

Best of luck!
Peter

 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Tuesday, February 26, 2013 9:20 AM
Subject: RE: Buckeye Sensor Model

 

Peter, of course!

I’m a little hesitant right now, because I think I’m trashing the stack somewhere in it…. I can ortho out to tifs fine, but for some reason when I try to use any plugin to write output, the application crashes at the return from write(). We’re also doing some extensive lens modeling here and I hope to get the correct distortion model hooked in as well – honestly I pretty much just copied the Applanix model as a start and went from there with our metadata format – still need to do the due-diligence and double check all of the math for my own sanity, but I am getting images projected to within 0.5-2m of our other processing methods.

 

-Andy

 

From: Peter Borissow [[hidden email]]
Sent: Tuesday, February 26, 2013 9:13 AM
To: Andrew D. Johnson
Cc: [hidden email]
Subject: Buckeye Sensor Model

 

Andrew-

    Would it be possible for you to share your Buckeye Sensor Model? It would be a great contribution to the ossim project.

 

The more people that can use Buckeye the better.

 

Peter

 

 

 


From: Andrew D. Johnson <[hidden email]>
To: Peter Borissow <[hidden email]>; Oscar L. Kramer <[hidden email]>; David Burken <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Sent: Monday, February 25, 2013 12:16 PM
Subject: RE: [OSSIM] Squeezing the most out of ossim

 

Buckeye imagery, I reworked the Sensor Model to work with the actual data that comes off the plane, (FrameMeta GSTI is not, but very similar). Pretty much I want to be able to ortho & mosaic ~3000 images in a somewhat efficient manner… (3-4k images is an average day of collection), each image is ~177mb, 8984x6732.

I was looking at ossim-mosaic, which looks like it might need to be updated to handle the other combiner-types (I like closestToCenter), I’m assuming that’s what it’s purpose in life is – let orthoigen do the ortho work, let osism-mosaic stitch things together.

 

 

 

From: Peter Borissow [[hidden email]]
Sent: Monday, February 25, 2013 12:07 PM
To: Andrew D. Johnson; Oscar L. Kramer; David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Squeezing the most out of ossim

 

What images are you ortho'ing?

 

This electronic communication and any attachments may contain confidential and proprietary information of DigitalGlobe, Inc. If you are not the intended recipient, or an agent or employee responsible for delivering this Communication to the intended recipient, or if you have received this Communication in error, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this communication in error, and delete the copy you received. DigitalGlobe reserves the right to monitor any electronic communication sent or received by its employees, agents or representatives.
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
1234