Retrieving Sub Datasets from an RPF's A.TOC file

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Retrieving Sub Datasets from an RPF's A.TOC file

Miller, Doug
I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.

------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?

Thanks for any help you can give,

Doug

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

David Burken
Hi Doug,

Ossim handles cib/cadrg data.  I can't say if it's faster or slower than
gdal.  You'd have to compare.  If you use ossim be sure to build
overviews.  It will treat each entry as a separate image.

Take care,
Dave


On 10/26/2015 04:51 PM, Miller, Doug wrote:

> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>
> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>
> Thanks for any help you can give,
>
> Doug
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper
In reply to this post by Miller, Doug
I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly

Sent from my iPad

> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>
> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>
> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>
> Thanks for any help you can give,
>
> Doug
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Garrett Potts-2
Hello Jim:

Hmmm,   I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.   I’ll poke at it to make sure.   Could have been some old min/max scanning code.


Take care


Garrett

> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>
> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>
> Sent from my iPad
>
>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>
>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>
>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>
>> Thanks for any help you can give,
>>
>> Doug
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Garrett Potts-2
Hello All:

The only search that happens in the CID/CADRG is to find the frames that intersect the request.   This looks pretty efficient except for the fact we do not cache frame sets.   I think the CIB/CADRG could be sped up considerably for nearby frames if there was a frame cache added.  

Jim,  there is an SQLITE standard now and we have a reader (thank you to Burken) that accepts sparse tilesets from an SQLITE database.   The standard is an OGC Spec for Geopackage support and has been pretty efficient for sets that we have tried.


Take care


Garrett

> On Oct 27, 2015, at 6:48 AM, Garrett Potts <[hidden email]> wrote:
>
> Hello Jim:
>
> Hmmm,   I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.   I’ll poke at it to make sure.   Could have been some old min/max scanning code.
>
>
> Take care
>
>
> Garrett
>
>> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>>
>> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>>
>> Sent from my iPad
>>
>>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>>
>>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>>
>>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>>
>>> Thanks for any help you can give,
>>>
>>> Doug
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> www.ossim.org
>>> Ossim-developer mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Miller, Doug
In reply to this post by James E. Hopper
Jim,

I believe that is the same behavior (opens each of the subfiles when you do a search) that GDAL has.  I've given thought to similar schemes including building individual GDAL ".vrt" files.  My hesitation was the issue of maintenance as new CADRG releases occur.

Doug
________________________________________
From: Jim Hopper <[hidden email]>
Sent: Tuesday, October 27, 2015 1:33 AM
To: Miller, Doug
Cc: [hidden email]
Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file

I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly

Sent from my iPad

> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>
> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>
> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>
> Thanks for any help you can give,
>
> Doug
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper
Doug,

The format will probably always rely on the subfiles, its the toc structure that might change, thats why i read it (and the eubfiles) using ossim which Garrett, Dave, et al keep up to date.  I would not worry to much about changing formats, once you have your index you only have to reindex if you update your map set so it doesn't have to be done much.  i used a structure that had a base path in the front of my index and then all the individual files were stored with relative path names.  then if i needed to move my software to a new machine, i only had to change the single base path and the rest of the index was unchanged. so i could just tar up our maps and move them to a new machine easily.


---- "Miller wrote:

> Jim,
>
> I believe that is the same behavior (opens each of the subfiles when you do a search) that GDAL has.  I've given thought to similar schemes including building individual GDAL ".vrt" files.  My hesitation was the issue of maintenance as new CADRG releases occur.
>
> Doug
> ________________________________________
> From: Jim Hopper <[hidden email]>
> Sent: Tuesday, October 27, 2015 1:33 AM
> To: Miller, Doug
> Cc: [hidden email]
> Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file
>
> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>
> Sent from my iPad
>
> > On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
> >
> > I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
> >
> > ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
> >
> > Thanks for any help you can give,
> >
> > Doug
> >
> > ------------------------------------------------------------------------------
> > _______________________________________________
> > www.ossim.org
> > Ossim-developer mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper
In reply to this post by Garrett Potts-2
Garrett,

i should have been more clear. The search was in my map program.  When the map was moved to a new location (say the user jumped to a new place) then you have a geographic rectangle that you need filled from the subfiles.  You search through the atoc to find the files that have rectangles inside this map rectangle and create a mosaic to display.  Problem is the atoc does not contain the rectangle for the subfiles so it has to go open up each file and read it's rectangle to tell if the file is in the base rectangle or not.  The CADRG data has a LOT of subfiles when  you combine the atoc for a large part of the world and so it has to open individually a lot of files to find the ones desired.  Add to that that the subfiles were stored on a network drive and not locally and it would become really slow to build the initial map in a new location.

I did this on a program a lot of years ago so a lot could have changed, but i don't see how this would since its part of the CADRG format.  We did implement a newer map program where the customer would not allow us to use this scheme they wanted the data on a wms server.  its not impossibly slow, but it is still much less slow than using my index was in the old program.


---- Garrett Potts <[hidden email]> wrote:
> Hello All:

The only search that happens in the CID/CADRG is to find the frames that intersect the request.   This looks pretty efficient except for the fact we do not cache frame sets.   I think the CIB/CADRG could be sped up considerably for nearby frames if there was a frame cache added.  

Jim,  there is an SQLITE standard now and we have a reader (thank you to Burken) that accepts sparse tilesets from an SQLITE database.   The standard is an OGC Spec for Geopackage support and has been pretty efficient for sets that we have tried.


Take care


Garrett

> On Oct 27, 2015, at 6:48 AM, Garrett Potts <[hidden email]> wrote:
>
> Hello Jim:
>
> Hmmm,   I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.   I’ll poke at it to make sure.   Could have been some old min/max scanning code.
>
>
> Take care
>
>
> Garrett
>
>> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>>
>> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>>
>> Sent from my iPad
>>
>>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>>
>>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>>
>>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>>
>>> Thanks for any help you can give,
>>>
>>> Doug
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> www.ossim.org
>>> Ossim-developer mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Miller, Doug
Jim,

Thanks for all the information.  

Here is a follow-up question.  GDAL breaks data down into sub-datasets (equivalent to a CADRG boundary).  In CADRG a boundary is a collection of frames (the number depends of the data series) and a frame is a 6x6 set of 256 x 256 bit sub-frames.   GDAL does not provide access to a particular frame, but I wonder if OSSIM allow accessing of individual frames.  So, does the OSSIM library provide access individual frames or does it only work at the boundary level?

Thanks,

Doug
________________________________________
From: [hidden email] <[hidden email]>
Sent: Tuesday, October 27, 2015 10:10 AM
To: Garrett Potts
Cc: Miller, Doug; ossim users
Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file

Garrett,

i should have been more clear. The search was in my map program.  When the map was moved to a new location (say the user jumped to a new place) then you have a geographic rectangle that you need filled from the subfiles.  You search through the atoc to find the files that have rectangles inside this map rectangle and create a mosaic to display.  Problem is the atoc does not contain the rectangle for the subfiles so it has to go open up each file and read it's rectangle to tell if the file is in the base rectangle or not.  The CADRG data has a LOT of subfiles when  you combine the atoc for a large part of the world and so it has to open individually a lot of files to find the ones desired.  Add to that that the subfiles were stored on a network drive and not locally and it would become really slow to build the initial map in a new location.

I did this on a program a lot of years ago so a lot could have changed, but i don't see how this would since its part of the CADRG format.  We did implement a newer map program where the customer would not allow us to use this scheme they wanted the data on a wms server.  its not impossibly slow, but it is still much less slow than using my index was in the old program.


---- Garrett Potts <[hidden email]> wrote:
> Hello All:

The only search that happens in the CID/CADRG is to find the frames that intersect the request.   This looks pretty efficient except for the fact we do not cache frame sets.   I think the CIB/CADRG could be sped up considerably for nearby frames if there was a frame cache added.

Jim,  there is an SQLITE standard now and we have a reader (thank you to Burken) that accepts sparse tilesets from an SQLITE database.   The standard is an OGC Spec for Geopackage support and has been pretty efficient for sets that we have tried.


Take care


Garrett

> On Oct 27, 2015, at 6:48 AM, Garrett Potts <[hidden email]> wrote:
>
> Hello Jim:
>
> Hmmm,   I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.   I’ll poke at it to make sure.   Could have been some old min/max scanning code.
>
>
> Take care
>
>
> Garrett
>
>> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>>
>> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>>
>> Sent from my iPad
>>
>>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>>
>>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>>
>>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>>
>>> Thanks for any help you can give,
>>>
>>> Doug
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> www.ossim.org
>>> Ossim-developer mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper-2
What I used is the OSSIM capability to read the data in a given rectangle. When I did it using the Atoc  I  did not see any good way to access files directly in OSSIM from the Atoc.

My index that replaced this just did a recursive scan of directories under the base and used OSSIM to get the boundary for each file. I then saved the records in subgroups ordered by locations. So I could find a subgroup from my search, that would narrow my search then I could jump to a specific area of the index and look for the individual files in the index. Once I had the list of files needed to cover the map rectangle I would create an OSSIM mosaic chain and use that to display map.

> On Oct 27, 2015, at 10:49 AM, Miller, Doug <[hidden email]> wrote:
>
> Jim,
>
> Thanks for all the information.  
>
> Here is a follow-up question.  GDAL breaks data down into sub-datasets (equivalent to a CADRG boundary).  In CADRG a boundary is a collection of frames (the number depends of the data series) and a frame is a 6x6 set of 256 x 256 bit sub-frames.   GDAL does not provide access to a particular frame, but I wonder if OSSIM allow accessing of individual frames.  So, does the OSSIM library provide access individual frames or does it only work at the boundary level?
>
> Thanks,
>
> Doug
> ________________________________________
> From: [hidden email] <[hidden email]>
> Sent: Tuesday, October 27, 2015 10:10 AM
> To: Garrett Potts
> Cc: Miller, Doug; ossim users
> Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file
>
> Garrett,
>
> i should have been more clear. The search was in my map program.  When the map was moved to a new location (say the user jumped to a new place) then you have a geographic rectangle that you need filled from the subfiles.  You search through the atoc to find the files that have rectangles inside this map rectangle and create a mosaic to display.  Problem is the atoc does not contain the rectangle for the subfiles so it has to go open up each file and read it's rectangle to tell if the file is in the base rectangle or not.  The CADRG data has a LOT of subfiles when  you combine the atoc for a large part of the world and so it has to open individually a lot of files to find the ones desired.  Add to that that the subfiles were stored on a network drive and not locally and it would become really slow to build the initial map in a new location.
>
> I did this on a program a lot of years ago so a lot could have changed, but i don't see how this would since its part of the CADRG format.  We did implement a newer map program where the customer would not allow us to use this scheme they wanted the data on a wms server.  its not impossibly slow, but it is still much less slow than using my index was in the old program.
>
>
> ---- Garrett Potts <[hidden email]> wrote:
>> Hello All:
>
> The only search that happens in the CID/CADRG is to find the frames that intersect the request.   This looks pretty efficient except for the fact we do not cache frame sets.   I think the CIB/CADRG could be sped up considerably for nearby frames if there was a frame cache added.
>
> Jim,  there is an SQLITE standard now and we have a reader (thank you to Burken) that accepts sparse tilesets from an SQLITE database.   The standard is an OGC Spec for Geopackage support and has been pretty efficient for sets that we have tried.
>
>
> Take care
>
>
> Garrett
>
>> On Oct 27, 2015, at 6:48 AM, Garrett Potts <[hidden email]> wrote:
>>
>> Hello Jim:
>>
>> Hmmm,   I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.   I’ll poke at it to make sure.   Could have been some old min/max scanning code.
>>
>>
>> Take care
>>
>>
>> Garrett
>>
>>> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>>>
>>> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>>>
>>> Sent from my iPad
>>>
>>>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>>>
>>>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>>>
>>>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>>>
>>>> Thanks for any help you can give,
>>>>
>>>> Doug
>>>>
>>>> ------------------------------------------------------------------------------
>>>> _______________________________________________
>>>> www.ossim.org
>>>> Ossim-developer mailing list
>>>> [hidden email]
>>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> www.ossim.org
>>> Ossim-developer mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Peter Borissow
Not sure if this will help but...

Several years ago we did a lot of work with RPF in OSSIM. Our primary goal was to manage and visualize *ALL* RPF data in a single web map service. The challenge was keeping up with all the updates from NGA. Whenever a new product update came out, we would look at individual tiles/cells and update our database at the cell level. When it came time to render the RPF cells, we would generate a ".rpf" file for a given AOI and feed it to orthoigen:


We used the same approach to manage/visualize other tiled data like DTED, SRTM, GDEM, USGS data, etc.

Peter



From: James E. Hopper <[hidden email]>
To: "Miller, Doug" <[hidden email]>
Cc: Garrett Potts <[hidden email]>; ossim users <[hidden email]>
Sent: Tuesday, October 27, 2015 11:41 AM
Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file

What I used is the OSSIM capability to read the data in a given rectangle. When I did it using the Atoc  I  did not see any good way to access files directly in OSSIM from the Atoc.

My index that replaced this just did a recursive scan of directories under the base and used OSSIM to get the boundary for each file. I then saved the records in subgroups ordered by locations. So I could find a subgroup from my search, that would narrow my search then I could jump to a specific area of the index and look for the individual files in the index. Once I had the list of files needed to cover the map rectangle I would create an OSSIM mosaic chain and use that to display map.

> On Oct 27, 2015, at 10:49 AM, Miller, Doug <[hidden email]> wrote:
>
> Jim,
>
> Thanks for all the information. 
>
> Here is a follow-up question.  GDAL breaks data down into sub-datasets (equivalent to a CADRG boundary).  In CADRG a boundary is a collection of frames (the number depends of the data series) and a frame is a 6x6 set of 256 x 256 bit sub-frames.  GDAL does not provide access to a particular frame, but I wonder if OSSIM allow accessing of individual frames.  So, does the OSSIM library provide access individual frames or does it only work at the boundary level?
>
> Thanks,
>
> Doug
> ________________________________________
> From: [hidden email] <[hidden email]>
> Sent: Tuesday, October 27, 2015 10:10 AM
> To: Garrett Potts
> Cc: Miller, Doug; ossim users
> Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file
>
> Garrett,
>
> i should have been more clear. The search was in my map program.  When the map was moved to a new location (say the user jumped to a new place) then you have a geographic rectangle that you need filled from the subfiles.  You search through the atoc to find the files that have rectangles inside this map rectangle and create a mosaic to display.  Problem is the atoc does not contain the rectangle for the subfiles so it has to go open up each file and read it's rectangle to tell if the file is in the base rectangle or not.  The CADRG data has a LOT of subfiles when  you combine the atoc for a large part of the world and so it has to open individually a lot of files to find the ones desired.  Add to that that the subfiles were stored on a network drive and not locally and it would become really slow to build the initial map in a new location.
>
> I did this on a program a lot of years ago so a lot could have changed, but i don't see how this would since its part of the CADRG format.  We did implement a newer map program where the customer would not allow us to use this scheme they wanted the data on a wms server.  its not impossibly slow, but it is still much less slow than using my index was in the old program.
>
>
> ---- Garrett Potts <[hidden email]> wrote:
>> Hello All:
>
> The only search that happens in the CID/CADRG is to find the frames that intersect the request.  This looks pretty efficient except for the fact we do not cache frame sets.  I think the CIB/CADRG could be sped up considerably for nearby frames if there was a frame cache added.
>
> Jim,  there is an SQLITE standard now and we have a reader (thank you to Burken) that accepts sparse tilesets from an SQLITE database.  The standard is an OGC Spec for Geopackage support and has been pretty efficient for sets that we have tried.
>
>
> Take care
>
>
> Garrett
>
>> On Oct 27, 2015, at 6:48 AM, Garrett Potts <[hidden email]> wrote:
>>
>> Hello Jim:
>>
>> Hmmm,  I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.  I’ll poke at it to make sure.  Could have been some old min/max scanning code.
>>
>>
>> Take care
>>
>>
>> Garrett
>>
>>> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>>>
>>> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.  I never did the entire world but did map large parts of it and was able to update very quickly
>>>
>>> Sent from my iPad
>>>
>>>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>>>
>>>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>>>
>>>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>>>
>>>> Thanks for any help you can give,
>>>>
>>>> Doug
>>>>
>>>> ------------------------------------------------------------------------------
>>>> _______________________________________________
>>>> www.ossim.org
>>>> Ossim-developer mailing list
>>>> [hidden email]
>>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer


>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> www.ossim.org
>>> Ossim-developer mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer



------------------------------------------------------------------------------

_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

David Burken
In reply to this post by James E. Hopper-2
Hi Doug,

Any ossim app, i.e. ossim-geocell, ossim-chipper, ossim-orthoigen will
open the using the  ossimCibCadrgTileSource.  Each entry will be treated
as an individual image.  The underlying tiles are hidden from you.  You
could test the speed by opening your image in ossim-geocell and zooming
around.  First do:

ossim-preproc -o --ch <your-a.toc>

So under the hood there is a ossimCibCadrgTileSource::getTile method
that will grab the sub-tiles it needs to fill whatever tile was requested.

Anything else you would have to code as Jim pointed out.

Hope that helps,
Dave


On 10/27/2015 11:41 AM, James E. Hopper wrote:

> What I used is the OSSIM capability to read the data in a given rectangle. When I did it using the Atoc  I  did not see any good way to access files directly in OSSIM from the Atoc.
>
> My index that replaced this just did a recursive scan of directories under the base and used OSSIM to get the boundary for each file. I then saved the records in subgroups ordered by locations. So I could find a subgroup from my search, that would narrow my search then I could jump to a specific area of the index and look for the individual files in the index. Once I had the list of files needed to cover the map rectangle I would create an OSSIM mosaic chain and use that to display map.
>
>> On Oct 27, 2015, at 10:49 AM, Miller, Doug <[hidden email]> wrote:
>>
>> Jim,
>>
>> Thanks for all the information.
>>
>> Here is a follow-up question.  GDAL breaks data down into sub-datasets (equivalent to a CADRG boundary).  In CADRG a boundary is a collection of frames (the number depends of the data series) and a frame is a 6x6 set of 256 x 256 bit sub-frames.   GDAL does not provide access to a particular frame, but I wonder if OSSIM allow accessing of individual frames.  So, does the OSSIM library provide access individual frames or does it only work at the boundary level?
>>
>> Thanks,
>>
>> Doug
>> ________________________________________
>> From: [hidden email] <[hidden email]>
>> Sent: Tuesday, October 27, 2015 10:10 AM
>> To: Garrett Potts
>> Cc: Miller, Doug; ossim users
>> Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file
>>
>> Garrett,
>>
>> i should have been more clear. The search was in my map program.  When the map was moved to a new location (say the user jumped to a new place) then you have a geographic rectangle that you need filled from the subfiles.  You search through the atoc to find the files that have rectangles inside this map rectangle and create a mosaic to display.  Problem is the atoc does not contain the rectangle for the subfiles so it has to go open up each file and read it's rectangle to tell if the file is in the base rectangle or not.  The CADRG data has a LOT of subfiles when  you combine the atoc for a large part of the world and so it has to open individually a lot of files to find the ones desired.  Add to that that the subfiles were stored on a network drive and not locally and it would become really slow to build the initial map in a new location.
>>
>> I did this on a program a lot of years ago so a lot could have changed, but i don't see how this would since its part of the CADRG format.  We did implement a newer map program where the customer would not allow us to use this scheme they wanted the data on a wms server.  its not impossibly slow, but it is still much less slow than using my index was in the old program.
>>
>>
>> ---- Garrett Potts <[hidden email]> wrote:
>>> Hello All:
>> The only search that happens in the CID/CADRG is to find the frames that intersect the request.   This looks pretty efficient except for the fact we do not cache frame sets.   I think the CIB/CADRG could be sped up considerably for nearby frames if there was a frame cache added.
>>
>> Jim,  there is an SQLITE standard now and we have a reader (thank you to Burken) that accepts sparse tilesets from an SQLITE database.   The standard is an OGC Spec for Geopackage support and has been pretty efficient for sets that we have tried.
>>
>>
>> Take care
>>
>>
>> Garrett
>>
>>> On Oct 27, 2015, at 6:48 AM, Garrett Potts <[hidden email]> wrote:
>>>
>>> Hello Jim:
>>>
>>> Hmmm,   I’ll have to look at the CIB/CADRB code.  If I remember correctly there should be no scanning.   I’ll poke at it to make sure.   Could have been some old min/max scanning code.
>>>
>>>
>>> Take care
>>>
>>>
>>> Garrett
>>>
>>>> On Oct 27, 2015, at 1:33 AM, Jim Hopper <[hidden email]> wrote:
>>>>
>>>> I built a moving map application with Ossim. When the dataset got large the times were completely unacceptable. It's been a while but the issue is it opens each of the subfiles when you do a search. I ended up using Ossim to build an index of all the files as one large data file then searched that and open the subfiles to draw directly. Basically I replaced the atoc with my index. This took a long time to build the index but ran really fast to display from the index.   I never did the entire world but did map large parts of it and was able to update very quickly
>>>>
>>>> Sent from my iPad
>>>>
>>>>> On Oct 26, 2015, at 4:51 PM, Miller, Doug <[hidden email]> wrote:
>>>>>
>>>>> I'm currently using GDAL.  When retrieving a sub dataset from GDAL using GDALOpen() and expressing the file path as  "NITF_TOC_ENTRY:CADRG_TLM50_50K_2_18:C:/rpf/a.toc "  there is a considerable delay.  This delay is acceptable if the RFP's coverage is for the United States, but if the coverage is for the world then it becomes a problem.
>>>>>
>>>>> ------>Is there a different way that I should be working with RPF (CADRG) within the GDAL framework or should I move on to OSSIM?
>>>>>
>>>>> Thanks for any help you can give,
>>>>>
>>>>> Doug
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> _______________________________________________
>>>>> www.ossim.org
>>>>> Ossim-developer mailing list
>>>>> [hidden email]
>>>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>>> ------------------------------------------------------------------------------
>>>> _______________________________________________
>>>> www.ossim.org
>>>> Ossim-developer mailing list
>>>> [hidden email]
>>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> www.ossim.org
>>> Ossim-developer mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> www.ossim.org
>> Ossim-developer mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ossim-developer
> ------------------------------------------------------------------------------
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer


------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper-2
David. Your method is what we used on the second map product. It's vastly slower than using a pre computed index. It's what I started the first map program using but the users HATED how slow the maps updated.

But testing it with more modern computers and updated OSSIM sounds like a great idea.

Best Jim

> On Oct 27, 2015, at 12:26 PM, David Burken <[hidden email]> wrote:
>
> Hi Doug,
>
> Any ossim app, i.e. ossim-geocell, ossim-chipper, ossim-orthoigen will
> open the using the  ossimCibCadrgTileSource.  Each entry will be treated
> as an individual image.  The underlying tiles are hidden from you.  You
> could test the speed by opening your image in ossim-geocell and zooming
> around.  First do:
>
> ossim-preproc -o --ch <your-a.toc>
>
> So under the hood there is a ossimCibCadrgTileSource::getTile method
> that will grab the sub-tiles it needs to fill whatever tile was requested.
>
> Anything else you would have to code as Jim pointed out.
>
> Hope that helps,
> Dave
>
>
>> On 10/27/2015 11:41 AM, James E. Hopper wrote:
>> What I used is the OSSIM capability to read the data in a given rectangle. When I did it using the Atoc  I  did not see any good way to access files directly in OSSIM from the Atoc.
>>
>> My index that replaced this just did a recursive scan of directories under the base and used OSSIM to get the boundary for each file. I then saved the records in subgroups ordered by locations. So I could find a subgroup from my search, that would narrow my search then I could jump to a specific area of the index and look for the individual files in the index. Once I had the list of files needed to cover the map rectangle I would create an OSSIM mosaic chain and use that to display map.
>>
>>> On Oct 27, 2015, at 10:49 AM, Miller, Doug <[hidden email]> wrote:
>>>
>>> Jim,
>>>
>>> Thanks for all the information.
>>>
>>> Here is a follow-up question.  GDAL breaks data down into sub-datasets (equivalent to a CADRG boundary).  In CADRG a boundary is a collection of frames (the number depends of the data series) and a frame is a 6x6 set of 256 x 256 bit sub-frames.   GDAL does not provide access to a particular frame, but I wonder if OSSIM allow accessing of individual frames.  So, does the OSSIM library provide access individual frames or does it only work at the boundary level?
>>>
>>> Thanks,
>>>
>>> Doug
>>> ________________________________________
>>> From: [hidden email] <[hidden email]>
>>> Sent: Tuesday, October 27, 2015 10:10 AM
>>> To: Garrett Potts
>>> Cc: Miller, Doug; ossim users
>>> Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file
>>>
>>> Garrett,
>>>
>>> i should have been more clear. The search was in my map program

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Miller, Doug
It seems as though RPFs were intended to cover smaller areas of the world, because from my experiences and what I'm hearing from others is that the "A.TOC" is always a performance bottleneck otherwise.  It makes sense because the target of this technology was originally for devices that had limited CPU.  When it moved to desktop applications where one would like to have one repository of data that covers the world the "A.TOC" mechanism cripples it.  Maybe that is why FalconView has their own scheme that does not use an "A.TOC".

Doug
________________________________________
From: James E. Hopper <[hidden email]>
Sent: Tuesday, October 27, 2015 5:03 PM
To: David Burken
Cc: [hidden email]
Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file

David. Your method is what we used on the second map product. It's vastly slower than using a pre computed index. It's what I started the first map program using but the users HATED how slow the maps updated.

But testing it with more modern computers and updated OSSIM sounds like a great idea.

Best Jim

> On Oct 27, 2015, at 12:26 PM, David Burken <[hidden email]> wrote:
>
> Hi Doug,
>
> Any ossim app, i.e. ossim-geocell, ossim-chipper, ossim-orthoigen will
> open the using the  ossimCibCadrgTileSource.  Each entry will be treated
> as an individual image.  The underlying tiles are hidden from you.  You
> could test the speed by opening your image in ossim-geocell and zooming
> around.  First do:
>
> ossim-preproc -o --ch <your-a.toc>
>
> So under the hood there is a ossimCibCadrgTileSource::getTile method
> that will grab the sub-tiles it needs to fill whatever tile was requested.
>
> Anything else you would have to code as Jim pointed out.
>
> Hope that helps,
> Dave
>
>
>> On 10/27/2015 11:41 AM, James E. Hopper wrote:
>> What I used is the OSSIM capability to read the data in a given rectangle. When I did it using the Atoc  I  did not see any good way to access files directly in OSSIM from the Atoc.
>>
>> My index that replaced this just did a recursive scan of directories under the base and used OSSIM to get the boundary for each file. I then saved the records in subgroups ordered by locations. So I could find a subgroup from my search, that would narrow my search then I could jump to a specific area of the index and look for the individual files in the index. Once I had the list of files needed to cover the map rectangle I would create an OSSIM mosaic chain and use that to display map.
>>
>>> On Oct 27, 2015, at 10:49 AM, Miller, Doug <[hidden email]> wrote:
>>>
>>> Jim,
>>>
>>> Thanks for all the information.
>>>
>>> Here is a follow-up question.  GDAL breaks data down into sub-datasets (equivalent to a CADRG boundary).  In CADRG a boundary is a collection of frames (the number depends of the data series) and a frame is a 6x6 set of 256 x 256 bit sub-frames.   GDAL does not provide access to a particular frame, but I wonder if OSSIM allow accessing of individual frames.  So, does the OSSIM library provide access individual frames or does it only work at the boundary level?
>>>
>>> Thanks,
>>>
>>> Doug
>>> ________________________________________
>>> From: [hidden email] <[hidden email]>
>>> Sent: Tuesday, October 27, 2015 10:10 AM
>>> To: Garrett Potts
>>> Cc: Miller, Doug; ossim users
>>> Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file
>>>
>>> Garrett,
>>>
>>> i should have been more clear. The search was in my map program

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper-2
In reply to this post by James E. Hopper-2
I have a question that I hope this august group might answer.  We need to grab imagery from a WMS server, but we want it at a specific resolution i.e. the resolution of the source imagery on the WMS server.  to get an image we need to supply the geographic and pixel bounds of the image which need to match the resolution of the source imagery if we want full resolution images.  The WMS spec didn't add to the get capabilities  response the resolution until the spec got up to version 1.3 and even then its optional.  lots of WMS servers out there do not supply this information such as GeoServer which is the one we are using.  To get around this we manually add the resolution as keywords in geoserver when we create a layer.  this is not a great solution as it means our product can only work with layers we have prepared.  is there any way to perhaps ask the geoserver for an image at its base resolution without knowing the resolution before hand?  Then we could just use OSSIM to give us the resolution from the image and all would be good.

thanks jim


------------------------------------------------------------------------------

_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

jratike80
In reply to this post by Miller, Doug

Hi,

 

If Geoserver is configured to serve raster layers also through WCS you can find the native resolution by sending DescribeCoverage. For example

http://demo.opengeo.org/geoserver/wcs?service=WCS&version=2.0.1&request=describecoverage&coverageid=usgs__nlcd

 

Look at the offsetVectors. But if WCS is available if also suits better than WMS for grabbing imagery.

 

-Jukka Rahkonen-

 

jim hopper wrote:

 

I have a question that I hope this august group might answer.  We need to grab imagery from a WMS server, but we want it at a specific resolution i.e. the resolution of the source imagery on the WMS server.  to get an image we need to supply the geographic and pixel bounds of the image which need to match the resolution of the source imagery if we want full resolution images.  The WMS spec didn't add to the get capabilities  response the resolution until the spec got up to version 1.3 and even then its optional.  lots of WMS servers out there do not supply this information such as GeoServer which is the one we are using.  To get around this we manually add the resolution as keywords in geoserver when we create a layer.  this is not a great solution as it means our product can only work with layers we have prepared.  is there any way to perhaps ask the geoserver for an image at its base resolution without knowing the resolution before hand?  Then we could just use OSSIM to give us the resolution from the image and all would be good.

 

thanks jim

 


------------------------------------------------------------------------------

_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

jratike80
In reply to this post by Miller, Doug

Hi,

 

It is indeed not obvious but I told to have a look at offserVectors

 

<gml:offsetVector srsName="http://www.opengis.net/def/crs/EPSG/0/5070">0.0 30.0</gml:offsetVector><gml:offsetVector srsName="http://www.opengis.net/def/crs/EPSG/0/5070">-30.0 0.0</gml:offsetVector>

 

First offset can be read “zero [meters] to [north], 30 [meters] to [east]. Another one defines that north coordinate is decreasing by 30 units. I closed the units in [brackets] because what units to use and what is the meaning of axis is defined with EPSG:5070 http://epsg-registry.org/export.htm?wkt=urn:ogc:def:crs:EPSG::5070.

 

As with all OGC services the first thing to do is GetCapabilities

http://demo.opengeo.org/geoserver/wfs?service=WCS&version=1.1.1&request=GetCapabilities

Coverages are listed with this version as Identifiers

<wcs:Identifier>Img_Sample</wcs:Identifier>

 

Then you know what to use in DescribeCoverage. You can’t know that the keyword is now “Identifiers” in plurals but that is written into the v. 1.1.1 standard.

http://demo.opengeo.org/geoserver/wfs?service=WCS&version=1.1.1&request=DescribeCoverage&Identifiers=Img_Sample

 

With this version you must interpret GridOffsets

<wcs:GridOffsets>0.07003690742624616 0.0 0.0 -0.05586772575250837</wcs:GridOffsets>

 

Notice the non-square pixels.

I have not written the WCS standards so the changing syntax in different versions is not my fault. However, all the information is available from the website or OGC.

 

-Jukka-

 

 

 

jim hopper

 

Thanks for the information. i looked at the data from your link, but i don't see anything thats obviously a resolution there?  I thought i had WCS turn on in our geoserver, but i can't make your url tailored to our server work. One issue is that geoserver (at least the version we have running) only s supports 1.0 and 1.1.1 of WCS, so i changed your URL to use 1.1.1.  I am assuming the string coverage_id is just the layer name?

 

On Thu, Nov 12, 2015 at 2:22 PM, Rahkonen Jukka (MML) <[hidden email]> wrote:

Hi,

 

If Geoserver is configured to serve raster layers also through WCS you can find the native resolution by sending DescribeCoverage. For example

http://demo.opengeo.org/geoserver/wcs?service=WCS&version=2.0.1&request=describecoverage&coverageid=usgs__nlcd

 

Look at the offsetVectors. But if WCS is available if also suits better than WMS for grabbing imagery.

 

-Jukka Rahkonen-

 

jim hopper wrote:

 

I have a question that I hope this august group might answer.  We need to grab imagery from a WMS server, but we want it at a specific resolution i.e. the resolution of the source imagery on the WMS server.  to get an image we need to supply the geographic and pixel bounds of the image which need to match the resolution of the source imagery if we want full resolution images.  The WMS spec didn't add to the get capabilities  response the resolution until the spec got up to version 1.3 and even then its optional.  lots of WMS servers out there do not supply this information such as GeoServer which is the one we are using.  To get around this we manually add the resolution as keywords in geoserver when we create a layer.  this is not a great solution as it means our product can only work with layers we have prepared.  is there any way to perhaps ask the geoserver for an image at its base resolution without knowing the resolution before hand?  Then we could just use OSSIM to give us the resolution from the image and all would be good.

 

thanks jim

 

 


------------------------------------------------------------------------------

_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper-2
In reply to this post by James E. Hopper-2
I downloaded and build source from git.  I built just the goal, png, and the openjpeg plugins for now as thats all i know.  the make install puts them in a directory lib64, but while ossim-info works, it doest seem to be finding the plugins.  i added the path to an older oasis_preferences.ini file but still not seeing them.

best jim


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

James E. Hopper-2
Sorry for bad title

> On Mar 2, 2016, at 4:02 PM, jim hopper <[hidden email]> wrote:
>
> I downloaded and build source from git.  I built just the goal, png, and the openjpeg plugins for now as thats all i know.  the make install puts them in a directory lib64, but while ossim-info works, it doest seem to be finding the plugins.  i added the path to an older oasis_preferences.ini file but still not seeing them.
>
> best jim
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> www.ossim.org
> Ossim-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ossim-developer

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
Reply | Threaded
Open this post in threaded view
|

Re: Retrieving Sub Datasets from an RPF's A.TOC file

Oscar Kramer
In reply to this post by James E. Hopper-2
Jim,

You have OSSIM_PREFS_FILE env var set to that ossim_preferences.ini? You verified that the old prefs file lists the plugins as being in "lib64", not the older "lib"?

Oscar



From: jim hopper <[hidden email]>
To:
Cc: ossim users <[hidden email]>
Sent: Wednesday, March 2, 2016 4:02 PM
Subject: Re: [OSSIM] Retrieving Sub Datasets from an RPF's A.TOC file

I downloaded and build source from git.  I built just the goal, png, and the openjpeg plugins for now as thats all i know.  the make install puts them in a directory lib64, but while ossim-info works, it doest seem to be finding the plugins.  i added the path to an older oasis_preferences.ini file but still not seeing them.

best jim


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
www.ossim.org
Ossim-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ossim-developer
12