[gdal-dev] Tiling aerial photos

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Tiling aerial photos

Paul Meems (Top-X)
Hi list,

I have several aerial photos and I use MapWindow GIS to view them.
MapWindow is already using GDAL v1.8

Instead of loading each aerial photo as an individual layer I want to create a local tiles store of the photos.
I can then just load the tiles I need based on the scale and view. This will most likely improve the performance.

The process will be something like this:
1. Get the filenames of the photos
2. Merge them into one
3. Create tiles
4. Put the tiles in a SQLite database

My first question: Is the above suggested process correct or should I do it differently?
My second question: What GDAL functions should I look into to accomplish my process?

Thanks,

Paul Meems
The Netherlands


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Even Rouault
Selon "Paul Meems (Top-X)" <[hidden email]>:

> Hi list,
>
> I have several aerial photos and I use MapWindow GIS to view them.
> MapWindow is already using GDAL v1.8
>
> Instead of loading each aerial photo as an individual layer I want to
> create a local tiles store of the photos.
> I can then just load the tiles I need based on the scale and view. This
> will most likely improve the performance.
>
> The process will be something like this:
> 1. Get the filenames of the photos
> 2. Merge them into one
> 3. Create tiles
> 4. Put the tiles in a SQLite database
>
> My first question: Is the above suggested process correct or should I do it
> differently?
> My second question: What GDAL functions should I look into to accomplish my
> process?

You could try to make a VRT from all your photos. It will be seen as a single
GDAL dataset, and will take care of the burden of opening the underlying photos
when needed. You can use the gdalbuildvrt utility to create the VRT from the
photos.

>
> Thanks,
>
> Paul Meems
> The Netherlands
>


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Paul Meems (Top-X)
Thanks Even,

http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
As I understand it, it will do the merging part (without actually merging).

But it doesn't do the tiling part, right? Or is the vrt so optimized tiling is no longer necessary?

Thanks,

Paul


2012/7/12 Even Rouault <[hidden email]>
Selon "Paul Meems (Top-X)" <[hidden email]>:

> Hi list,
>
> I have several aerial photos and I use MapWindow GIS to view them.
> MapWindow is already using GDAL v1.8
>
> Instead of loading each aerial photo as an individual layer I want to
> create a local tiles store of the photos.
> I can then just load the tiles I need based on the scale and view. This
> will most likely improve the performance.
>
> The process will be something like this:
> 1. Get the filenames of the photos
> 2. Merge them into one
> 3. Create tiles
> 4. Put the tiles in a SQLite database
>
> My first question: Is the above suggested process correct or should I do it
> differently?
> My second question: What GDAL functions should I look into to accomplish my
> process?

You could try to make a VRT from all your photos. It will be seen as a single
GDAL dataset, and will take care of the burden of opening the underlying photos
when needed. You can use the gdalbuildvrt utility to create the VRT from the
photos.

>
> Thanks,
>
> Paul Meems
> The Netherlands
>


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Even Rouault
Selon "Paul Meems (Top-X)" <[hidden email]>:

> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).

The VRT driver will do on-the-fly merging of tiles that have overlapping. The
VRT itself is just a XML file.

>
> But it doesn't do the tiling part, right?

No, I wasn't sure if your photos were already regularly tiled or not. Note that
the VRT accepts non regularly tiled images. They can have overlapping, gaps,
different resolutions, etc. The main constraints are :
- they are in the same projection
- they are "north-up", that is to say there is no rotation or skewing term in
their geotransform matrix
- they have the same number of bands

> Or is the vrt so optimized tiling
> is no longer necessary?

Not necessary. Note that the VRT has no internal spatial indexing, so if you
have several dozains of thousands of images in a VRT, it might slow down because
it will iterate over all the image descriptions (without needing to open them
however, all the information is in the VRT) to see if they intersect with the
request window. But I'd expect the number of images in the VRT to be really high
for that effect to become noticeable.

>
> Thanks,
>
> Paul
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Dmitry Baryshnikov
In reply to this post by Paul Meems (Top-X)
12.07.2012 14:36, Paul Meems (Top-X) пишет:
Thanks Even,

http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
As I understand it, it will do the merging part (without actually merging).

But it doesn't do the tiling part, right? Or is the vrt so optimized tiling is no longer necessary?

Thanks,

Paul


2012/7/12 Even Rouault <[hidden email]>
Selon "Paul Meems (Top-X)" <[hidden email]>:

> Hi list,
>
> I have several aerial photos and I use MapWindow GIS to view them.
> MapWindow is already using GDAL v1.8
>
> Instead of loading each aerial photo as an individual layer I want to
> create a local tiles store of the photos.
> I can then just load the tiles I need based on the scale and view. This
> will most likely improve the performance.
>
> The process will be something like this:
> 1. Get the filenames of the photos
> 2. Merge them into one
> 3. Create tiles
> 4. Put the tiles in a SQLite database
>
> My first question: Is the above suggested process correct or should I do it
> differently?
> My second question: What GDAL functions should I look into to accomplish my
> process?

You could try to make a VRT from all your photos. It will be seen as a single
GDAL dataset, and will take care of the burden of opening the underlying photos
when needed. You can use the gdalbuildvrt utility to create the VRT from the
photos.

>
> Thanks,
>
> Paul Meems
> The Netherlands
>


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev



_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
There is an interesting work connected aerial imagery: http://trac.osgeo.org/gdal/wiki/Correlator
http://correlatorgsoc2012.blogspot.com/

Best regards,
    Dmitry

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Paul Meems (Top-X)
Thanks Dmitry and Even,

The aerial photos are north-up and are in the same projection and I think also in the same resolution.
It are commercial aerial photos.

The Correlator project sounds very interesting but not necessary in my case.
I'll try to implement the VRT format and see what the performance will be.
If it is fast we don't need to tile.

Thanks,

Paul

2012/7/12 Even Rouault <[hidden email]>
Selon "Paul Meems (Top-X)" <[hidden email]>:

> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).

The VRT driver will do on-the-fly merging of tiles that have overlapping. The
VRT itself is just a XML file.

>
> But it doesn't do the tiling part, right?

No, I wasn't sure if your photos were already regularly tiled or not. Note that
the VRT accepts non regularly tiled images. They can have overlapping, gaps,
different resolutions, etc. The main constraints are :
- they are in the same projection
- they are "north-up", that is to say there is no rotation or skewing term in
their geotransform matrix
- they have the same number of bands

> Or is the vrt so optimized tiling
> is no longer necessary?

Not necessary. Note that the VRT has no internal spatial indexing, so if you
have several dozains of thousands of images in a VRT, it might slow down because
it will iterate over all the image descriptions (without needing to open them
however, all the information is in the VRT) to see if they intersect with the
request window. But I'd expect the number of images in the VRT to be really high
for that effect to become noticeable.

>
> Thanks,
>
> Paul

2012/7/12 Dmitry Baryshnikov <[hidden email]>
12.07.2012 14:36, Paul Meems (Top-X) пишет:
Thanks Even,

http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
As I understand it, it will do the merging part (without actually merging).

But it doesn't do the tiling part, right? Or is the vrt so optimized tiling is no longer necessary?

Thanks,

Paul


2012/7/12 Even Rouault <[hidden email]>
Selon "Paul Meems (Top-X)" <[hidden email]>:

> Hi list,
>
> I have several aerial photos and I use MapWindow GIS to view them.
> MapWindow is already using GDAL v1.8
>
> Instead of loading each aerial photo as an individual layer I want to
> create a local tiles store of the photos.
> I can then just load the tiles I need based on the scale and view. This
> will most likely improve the performance.
>
> The process will be something like this:
> 1. Get the filenames of the photos
> 2. Merge them into one
> 3. Create tiles
> 4. Put the tiles in a SQLite database
>
> My first question: Is the above suggested process correct or should I do it
> differently?
> My second question: What GDAL functions should I look into to accomplish my
> process?

You could try to make a VRT from all your photos. It will be seen as a single
GDAL dataset, and will take care of the burden of opening the underlying photos
when needed. You can use the gdalbuildvrt utility to create the VRT from the
photos.

>
> Thanks,
>
> Paul Meems
> The Netherlands
>


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev



_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
There is an interesting work connected aerial imagery: http://trac.osgeo.org/gdal/wiki/Correlator
http://correlatorgsoc2012.blogspot.com/

Best regards,
    Dmitry

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

dncarreira

Hi Paul.

 

I had to build a big vrt. The one trick that got it loading very fast into qgis was adding statistics to each rasterband:

  <Metadata>

      <MDI key="STATISTICS_MAXIMUM">255</MDI>

      <MDI key="STATISTICS_MEAN">111.6784426525</MDI>

      <MDI key="STATISTICS_MINIMUM">0</MDI>

      <MDI key="STATISTICS_STDDEV">52.100074055799</MDI>

    </Metadata>

 

I just got stats from a subset and used that for all bands.

 

Duarte

 

De: Paul Meems (Top-X) [mailto:[hidden email]]
Enviada: quinta-feira, 12 de Julho de 2012 12:11
Para: [hidden email]
Assunto: Re: [gdal-dev] Tiling aerial photos

 

Thanks Dmitry and Even,

The aerial photos are north-up and are in the same projection and I think also in the same resolution.
It are commercial aerial photos.

The Correlator project sounds very interesting but not necessary in my case.
I'll try to implement the VRT format and see what the performance will be.
If it is fast we don't need to tile.

Thanks,

Paul

2012/7/12 Even Rouault <[hidden email]>

Selon "Paul Meems (Top-X)" <[hidden email]>:

> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).

The VRT driver will do on-the-fly merging of tiles that have overlapping. The
VRT itself is just a XML file.


>
> But it doesn't do the tiling part, right?

No, I wasn't sure if your photos were already regularly tiled or not. Note that
the VRT accepts non regularly tiled images. They can have overlapping, gaps,
different resolutions, etc. The main constraints are :
- they are in the same projection
- they are "north-up", that is to say there is no rotation or skewing term in
their geotransform matrix
- they have the same number of bands


> Or is the vrt so optimized tiling
> is no longer necessary?

Not necessary. Note that the VRT has no internal spatial indexing, so if you
have several dozains of thousands of images in a VRT, it might slow down because
it will iterate over all the image descriptions (without needing to open them
however, all the information is in the VRT) to see if they intersect with the
request window. But I'd expect the number of images in the VRT to be really high
for that effect to become noticeable.

>
> Thanks,
>
> Paul

2012/7/12 Dmitry Baryshnikov <[hidden email]>

12.07.2012 14:36, Paul Meems (Top-X) пишет:

Thanks Even,

http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
As I understand it, it will do the merging part (without actually merging).

But it doesn't do the tiling part, right? Or is the vrt so optimized tiling is no longer necessary?

Thanks,

Paul

2012/7/12 Even Rouault <[hidden email]>

Selon "Paul Meems (Top-X)" <[hidden email]>:


> Hi list,
>
> I have several aerial photos and I use MapWindow GIS to view them.
> MapWindow is already using GDAL v1.8
>
> Instead of loading each aerial photo as an individual layer I want to
> create a local tiles store of the photos.
> I can then just load the tiles I need based on the scale and view. This
> will most likely improve the performance.
>
> The process will be something like this:
> 1. Get the filenames of the photos
> 2. Merge them into one
> 3. Create tiles
> 4. Put the tiles in a SQLite database
>
> My first question: Is the above suggested process correct or should I do it
> differently?
> My second question: What GDAL functions should I look into to accomplish my
> process?

You could try to make a VRT from all your photos. It will be seen as a single
GDAL dataset, and will take care of the burden of opening the underlying photos
when needed. You can use the gdalbuildvrt utility to create the VRT from the
photos.


>
> Thanks,
>
> Paul Meems
> The Netherlands
>

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev



_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev

There is an interesting work connected aerial imagery: http://trac.osgeo.org/gdal/wiki/Correlator
http://correlatorgsoc2012.blogspot.com/

Best regards,
    Dmitry


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev

 


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Etienne Tourigny-3
You should generate the stats for every file, as you will probably get
incorrect results with your method

for f in *.tif; do gdalinfo -stats $f ; done

On Thu, Jul 12, 2012 at 9:03 AM, Duarte Carreira <[hidden email]> wrote:

> Hi Paul.
>
>
>
> I had to build a big vrt. The one trick that got it loading very fast into
> qgis was adding statistics to each rasterband:
>
>   <Metadata>
>
>       <MDI key="STATISTICS_MAXIMUM">255</MDI>
>
>       <MDI key="STATISTICS_MEAN">111.6784426525</MDI>
>
>       <MDI key="STATISTICS_MINIMUM">0</MDI>
>
>       <MDI key="STATISTICS_STDDEV">52.100074055799</MDI>
>
>     </Metadata>
>
>
>
> I just got stats from a subset and used that for all bands.
>
>
>
> Duarte
>
>
>
> De: Paul Meems (Top-X) [mailto:[hidden email]]
> Enviada: quinta-feira, 12 de Julho de 2012 12:11
> Para: [hidden email]
> Assunto: Re: [gdal-dev] Tiling aerial photos
>
>
>
> Thanks Dmitry and Even,
>
> The aerial photos are north-up and are in the same projection and I think
> also in the same resolution.
> It are commercial aerial photos.
>
> The Correlator project sounds very interesting but not necessary in my case.
> I'll try to implement the VRT format and see what the performance will be.
> If it is fast we don't need to tile.
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>> Thanks Even,
>>
>> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
>> As I understand it, it will do the merging part (without actually
>> merging).
>
> The VRT driver will do on-the-fly merging of tiles that have overlapping.
> The
> VRT itself is just a XML file.
>
>
>>
>> But it doesn't do the tiling part, right?
>
> No, I wasn't sure if your photos were already regularly tiled or not. Note
> that
> the VRT accepts non regularly tiled images. They can have overlapping, gaps,
> different resolutions, etc. The main constraints are :
> - they are in the same projection
> - they are "north-up", that is to say there is no rotation or skewing term
> in
> their geotransform matrix
> - they have the same number of bands
>
>
>> Or is the vrt so optimized tiling
>> is no longer necessary?
>
> Not necessary. Note that the VRT has no internal spatial indexing, so if you
> have several dozains of thousands of images in a VRT, it might slow down
> because
> it will iterate over all the image descriptions (without needing to open
> them
> however, all the information is in the VRT) to see if they intersect with
> the
> request window. But I'd expect the number of images in the VRT to be really
> high
> for that effect to become noticeable.
>
>>
>> Thanks,
>>
>> Paul
>
> 2012/7/12 Dmitry Baryshnikov <[hidden email]>
>
> 12.07.2012 14:36, Paul Meems (Top-X) пишет:
>
> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).
>
> But it doesn't do the tiling part, right? Or is the vrt so optimized tiling
> is no longer necessary?
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>
>> Hi list,
>>
>> I have several aerial photos and I use MapWindow GIS to view them.
>> MapWindow is already using GDAL v1.8
>>
>> Instead of loading each aerial photo as an individual layer I want to
>> create a local tiles store of the photos.
>> I can then just load the tiles I need based on the scale and view. This
>> will most likely improve the performance.
>>
>> The process will be something like this:
>> 1. Get the filenames of the photos
>> 2. Merge them into one
>> 3. Create tiles
>> 4. Put the tiles in a SQLite database
>>
>> My first question: Is the above suggested process correct or should I do
>> it
>> differently?
>> My second question: What GDAL functions should I look into to accomplish
>> my
>> process?
>
> You could try to make a VRT from all your photos. It will be seen as a
> single
> GDAL dataset, and will take care of the burden of opening the underlying
> photos
> when needed. You can use the gdalbuildvrt utility to create the VRT from the
> photos.
>
>
>>
>> Thanks,
>>
>> Paul Meems
>> The Netherlands
>>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
>
> gdal-dev mailing list
>
> [hidden email]
>
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> There is an interesting work connected aerial imagery:
> http://trac.osgeo.org/gdal/wiki/Correlator
> http://correlatorgsoc2012.blogspot.com/
>
> Best regards,
>     Dmitry
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Paul Meems (Top-X)
Thanks Duarte and Etienne,

That is very useful information.
I wouldn't thought about it reading just the documentation.

Thanks,


Paul


2012/7/12 Etienne Tourigny <[hidden email]>
You should generate the stats for every file, as you will probably get
incorrect results with your method

for f in *.tif; do gdalinfo -stats $f ; done

On Thu, Jul 12, 2012 at 9:03 AM, Duarte Carreira <[hidden email]> wrote:
> Hi Paul.
>
>
>
> I had to build a big vrt. The one trick that got it loading very fast into
> qgis was adding statistics to each rasterband:
>
>   <Metadata>
>
>       <MDI key="STATISTICS_MAXIMUM">255</MDI>
>
>       <MDI key="STATISTICS_MEAN"><a href="tel:111.6784426525" value="+16784426525">111.6784426525</MDI>
>
>       <MDI key="STATISTICS_MINIMUM">0</MDI>
>
>       <MDI key="STATISTICS_STDDEV">52.100074055799</MDI>
>
>     </Metadata>
>
>
>
> I just got stats from a subset and used that for all bands.
>
>
>
> Duarte
>
>
>
> De: Paul Meems (Top-X) [mailto:[hidden email]]
> Enviada: quinta-feira, 12 de Julho de 2012 12:11
> Para: [hidden email]
> Assunto: Re: [gdal-dev] Tiling aerial photos
>
>
>
> Thanks Dmitry and Even,
>
> The aerial photos are north-up and are in the same projection and I think
> also in the same resolution.
> It are commercial aerial photos.
>
> The Correlator project sounds very interesting but not necessary in my case.
> I'll try to implement the VRT format and see what the performance will be.
> If it is fast we don't need to tile.
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>> Thanks Even,
>>
>> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
>> As I understand it, it will do the merging part (without actually
>> merging).
>
> The VRT driver will do on-the-fly merging of tiles that have overlapping.
> The
> VRT itself is just a XML file.
>
>
>>
>> But it doesn't do the tiling part, right?
>
> No, I wasn't sure if your photos were already regularly tiled or not. Note
> that
> the VRT accepts non regularly tiled images. They can have overlapping, gaps,
> different resolutions, etc. The main constraints are :
> - they are in the same projection
> - they are "north-up", that is to say there is no rotation or skewing term
> in
> their geotransform matrix
> - they have the same number of bands
>
>
>> Or is the vrt so optimized tiling
>> is no longer necessary?
>
> Not necessary. Note that the VRT has no internal spatial indexing, so if you
> have several dozains of thousands of images in a VRT, it might slow down
> because
> it will iterate over all the image descriptions (without needing to open
> them
> however, all the information is in the VRT) to see if they intersect with
> the
> request window. But I'd expect the number of images in the VRT to be really
> high
> for that effect to become noticeable.
>
>>
>> Thanks,
>>
>> Paul
>
> 2012/7/12 Dmitry Baryshnikov <[hidden email]>
>
> 12.07.2012 14:36, Paul Meems (Top-X) пишет:
>
> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).
>
> But it doesn't do the tiling part, right? Or is the vrt so optimized tiling
> is no longer necessary?
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>
>> Hi list,
>>
>> I have several aerial photos and I use MapWindow GIS to view them.
>> MapWindow is already using GDAL v1.8
>>
>> Instead of loading each aerial photo as an individual layer I want to
>> create a local tiles store of the photos.
>> I can then just load the tiles I need based on the scale and view. This
>> will most likely improve the performance.
>>
>> The process will be something like this:
>> 1. Get the filenames of the photos
>> 2. Merge them into one
>> 3. Create tiles
>> 4. Put the tiles in a SQLite database
>>
>> My first question: Is the above suggested process correct or should I do
>> it
>> differently?
>> My second question: What GDAL functions should I look into to accomplish
>> my
>> process?
>
> You could try to make a VRT from all your photos. It will be seen as a
> single
> GDAL dataset, and will take care of the burden of opening the underlying
> photos
> when needed. You can use the gdalbuildvrt utility to create the VRT from the
> photos.
>
>
>>
>> Thanks,
>>
>> Paul Meems
>> The Netherlands
>>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
>
> gdal-dev mailing list
>
> [hidden email]
>
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> There is an interesting work connected aerial imagery:
> http://trac.osgeo.org/gdal/wiki/Correlator
> http://correlatorgsoc2012.blogspot.com/
>
> Best regards,
>     Dmitry
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

dncarreira
In reply to this post by Etienne Tourigny-3
For some reason stats in the individual files did not solve the issue, had to put it in the vrt.

Duarte

-----Mensagem original-----
De: Etienne Tourigny [mailto:[hidden email]]
Enviada: quinta-feira, 12 de Julho de 2012 13:28
Para: Duarte Carreira
Cc: [hidden email]; [hidden email]
Assunto: Re: [gdal-dev] Tiling aerial photos

You should generate the stats for every file, as you will probably get incorrect results with your method

for f in *.tif; do gdalinfo -stats $f ; done

On Thu, Jul 12, 2012 at 9:03 AM, Duarte Carreira <[hidden email]> wrote:

> Hi Paul.
>
>
>
> I had to build a big vrt. The one trick that got it loading very fast
> into qgis was adding statistics to each rasterband:
>
>   <Metadata>
>
>       <MDI key="STATISTICS_MAXIMUM">255</MDI>
>
>       <MDI key="STATISTICS_MEAN">111.6784426525</MDI>
>
>       <MDI key="STATISTICS_MINIMUM">0</MDI>
>
>       <MDI key="STATISTICS_STDDEV">52.100074055799</MDI>
>
>     </Metadata>
>
>
>
> I just got stats from a subset and used that for all bands.
>
>
>
> Duarte
>
>
>
> De: Paul Meems (Top-X) [mailto:[hidden email]]
> Enviada: quinta-feira, 12 de Julho de 2012 12:11
> Para: [hidden email]
> Assunto: Re: [gdal-dev] Tiling aerial photos
>
>
>
> Thanks Dmitry and Even,
>
> The aerial photos are north-up and are in the same projection and I
> think also in the same resolution.
> It are commercial aerial photos.
>
> The Correlator project sounds very interesting but not necessary in my case.
> I'll try to implement the VRT format and see what the performance will be.
> If it is fast we don't need to tile.
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>> Thanks Even,
>>
>> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
>> As I understand it, it will do the merging part (without actually
>> merging).
>
> The VRT driver will do on-the-fly merging of tiles that have overlapping.
> The
> VRT itself is just a XML file.
>
>
>>
>> But it doesn't do the tiling part, right?
>
> No, I wasn't sure if your photos were already regularly tiled or not.
> Note that the VRT accepts non regularly tiled images. They can have
> overlapping, gaps, different resolutions, etc. The main constraints
> are :
> - they are in the same projection
> - they are "north-up", that is to say there is no rotation or skewing
> term in their geotransform matrix
> - they have the same number of bands
>
>
>> Or is the vrt so optimized tiling
>> is no longer necessary?
>
> Not necessary. Note that the VRT has no internal spatial indexing, so
> if you have several dozains of thousands of images in a VRT, it might
> slow down because it will iterate over all the image descriptions
> (without needing to open them however, all the information is in the
> VRT) to see if they intersect with the request window. But I'd expect
> the number of images in the VRT to be really high for that effect to
> become noticeable.
>
>>
>> Thanks,
>>
>> Paul
>
> 2012/7/12 Dmitry Baryshnikov <[hidden email]>
>
> 12.07.2012 14:36, Paul Meems (Top-X) пишет:
>
> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).
>
> But it doesn't do the tiling part, right? Or is the vrt so optimized
> tiling is no longer necessary?
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>
>> Hi list,
>>
>> I have several aerial photos and I use MapWindow GIS to view them.
>> MapWindow is already using GDAL v1.8
>>
>> Instead of loading each aerial photo as an individual layer I want to
>> create a local tiles store of the photos.
>> I can then just load the tiles I need based on the scale and view.
>> This will most likely improve the performance.
>>
>> The process will be something like this:
>> 1. Get the filenames of the photos
>> 2. Merge them into one
>> 3. Create tiles
>> 4. Put the tiles in a SQLite database
>>
>> My first question: Is the above suggested process correct or should I
>> do it differently?
>> My second question: What GDAL functions should I look into to
>> accomplish my process?
>
> You could try to make a VRT from all your photos. It will be seen as a
> single GDAL dataset, and will take care of the burden of opening the
> underlying photos when needed. You can use the gdalbuildvrt utility to
> create the VRT from the photos.
>
>
>>
>> Thanks,
>>
>> Paul Meems
>> The Netherlands
>>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
>
> gdal-dev mailing list
>
> [hidden email]
>
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> There is an interesting work connected aerial imagery:
> http://trac.osgeo.org/gdal/wiki/Correlator
> http://correlatorgsoc2012.blogspot.com/
>
> Best regards,
>     Dmitry
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Paul Meems (Top-X)
One more related question.
It seems using the VRT format is the way to go.

Of course I can make a wrapper around the executable and create a VRT like that.
I can also add the functionality to the MapWindow ActiveX control, it is already using GDAL for lots of other stuff.
Is gdalbuildvrt.exe easily included in another application or should I stick with the executable?

Does anybody know how QGis is doing it?

Thanks,

Paul


2012/7/12 Duarte Carreira <[hidden email]>
For some reason stats in the individual files did not solve the issue, had to put it in the vrt.

Duarte

-----Mensagem original-----
De: Etienne Tourigny [mailto:[hidden email]]
Enviada: quinta-feira, 12 de Julho de 2012 13:28
Para: Duarte Carreira
Cc: [hidden email]; [hidden email]
Assunto: Re: [gdal-dev] Tiling aerial photos

You should generate the stats for every file, as you will probably get incorrect results with your method

for f in *.tif; do gdalinfo -stats $f ; done

On Thu, Jul 12, 2012 at 9:03 AM, Duarte Carreira <[hidden email]> wrote:
> Hi Paul.
>
>
>
> I had to build a big vrt. The one trick that got it loading very fast
> into qgis was adding statistics to each rasterband:
>
>   <Metadata>
>
>       <MDI key="STATISTICS_MAXIMUM">255</MDI>
>
>       <MDI key="STATISTICS_MEAN"><a href="tel:111.6784426525" value="+16784426525">111.6784426525</MDI>
>
>       <MDI key="STATISTICS_MINIMUM">0</MDI>
>
>       <MDI key="STATISTICS_STDDEV">52.100074055799</MDI>
>
>     </Metadata>
>
>
>
> I just got stats from a subset and used that for all bands.
>
>
>
> Duarte
>
>
>
> De: Paul Meems (Top-X) [mailto:[hidden email]]
> Enviada: quinta-feira, 12 de Julho de 2012 12:11
> Para: [hidden email]
> Assunto: Re: [gdal-dev] Tiling aerial photos
>
>
>
> Thanks Dmitry and Even,
>
> The aerial photos are north-up and are in the same projection and I
> think also in the same resolution.
> It are commercial aerial photos.
>
> The Correlator project sounds very interesting but not necessary in my case.
> I'll try to implement the VRT format and see what the performance will be.
> If it is fast we don't need to tile.
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>> Thanks Even,
>>
>> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
>> As I understand it, it will do the merging part (without actually
>> merging).
>
> The VRT driver will do on-the-fly merging of tiles that have overlapping.
> The
> VRT itself is just a XML file.
>
>
>>
>> But it doesn't do the tiling part, right?
>
> No, I wasn't sure if your photos were already regularly tiled or not.
> Note that the VRT accepts non regularly tiled images. They can have
> overlapping, gaps, different resolutions, etc. The main constraints
> are :
> - they are in the same projection
> - they are "north-up", that is to say there is no rotation or skewing
> term in their geotransform matrix
> - they have the same number of bands
>
>
>> Or is the vrt so optimized tiling
>> is no longer necessary?
>
> Not necessary. Note that the VRT has no internal spatial indexing, so
> if you have several dozains of thousands of images in a VRT, it might
> slow down because it will iterate over all the image descriptions
> (without needing to open them however, all the information is in the
> VRT) to see if they intersect with the request window. But I'd expect
> the number of images in the VRT to be really high for that effect to
> become noticeable.
>
>>
>> Thanks,
>>
>> Paul
>
> 2012/7/12 Dmitry Baryshnikov <[hidden email]>
>
> 12.07.2012 14:36, Paul Meems (Top-X) пишет:
>
> Thanks Even,
>
> http://www.gdal.org/gdalbuildvrt.html does seems very interesting.
> As I understand it, it will do the merging part (without actually merging).
>
> But it doesn't do the tiling part, right? Or is the vrt so optimized
> tiling is no longer necessary?
>
> Thanks,
>
> Paul
>
> 2012/7/12 Even Rouault <[hidden email]>
>
> Selon "Paul Meems (Top-X)" <[hidden email]>:
>
>
>> Hi list,
>>
>> I have several aerial photos and I use MapWindow GIS to view them.
>> MapWindow is already using GDAL v1.8
>>
>> Instead of loading each aerial photo as an individual layer I want to
>> create a local tiles store of the photos.
>> I can then just load the tiles I need based on the scale and view.
>> This will most likely improve the performance.
>>
>> The process will be something like this:
>> 1. Get the filenames of the photos
>> 2. Merge them into one
>> 3. Create tiles
>> 4. Put the tiles in a SQLite database
>>
>> My first question: Is the above suggested process correct or should I
>> do it differently?
>> My second question: What GDAL functions should I look into to
>> accomplish my process?
>
> You could try to make a VRT from all your photos. It will be seen as a
> single GDAL dataset, and will take care of the burden of opening the
> underlying photos when needed. You can use the gdalbuildvrt utility to
> create the VRT from the photos.
>
>
>>
>> Thanks,
>>
>> Paul Meems
>> The Netherlands
>>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
>
> gdal-dev mailing list
>
> [hidden email]
>
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> There is an interesting work connected aerial imagery:
> http://trac.osgeo.org/gdal/wiki/Correlator
> http://correlatorgsoc2012.blogspot.com/
>
> Best regards,
>     Dmitry
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tiling aerial photos

Even Rouault
Selon "Paul Meems (Top-X)" <[hidden email]>:

> One more related question.
> It seems using the VRT format is the way to go.
>
> Of course I can make a wrapper around the executable and create a VRT like
> that.
> I can also add the functionality to the MapWindow ActiveX control, it is
> already using GDAL for lots of other stuff.
> Is gdalbuildvrt.exe easily included in another application or should I
> stick with the executable?

If you don't want to call gdalbuildvrt, embedding its source code into your app
should be doable. It just uses the public API of GDAL to do 99% of its job. It
is just r20960 that introduced a dependency on the C++ VRTDataset class for the
advanced functionnality of handling mask bands. You could remove it if you want
to stick to the C API only.

>
> Does anybody know how QGis is doing it?

QGIS will just manipulate the VRT as any other GDAL raster format, without
knowing it is a VRT. And for VRT creation, the GDALTools plugin will call
gdalbuildvrt.

>
> Thanks,
>
> Paul
>
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev