[gdal-dev] Tilesize in JPEG2000 and gdalinfo

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Tilesize in JPEG2000 and gdalinfo

jratike80

Hi,

 

If I run gdalinfo against the JPEG2000 image in http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m/1/N5144H.jp2

the JP2ECW driver reports the blocksize as (256x256) and JPEG2000OpenJPEG driver reports that the block size is (1024x1024).

 

What interests me would be to know the fact that this image has only one JPEG2000 tile with a tilesize of (11754x11754). Is there any driver-independent way for getting this information?

 

-Jukka Rahkonen-


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tilesize in JPEG2000 and gdalinfo

Even Rouault-2

Hi Jukka,

 

>

> If I run gdalinfo against the JPEG2000 image in

> http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m

> /1/N5144H.jp2 the JP2ECW driver reports the blocksize as (256x256) and

> JPEG2000OpenJPEG driver reports that the block size is (1024x1024).

 

Yes for single tiled images, all JPEG2000 drivers will report a block size

of much smaller dimension to avoid GDAL consuming too much memory.

 

>

> What interests me would be to know the fact that this image has only one

> JPEG2000 tile with a tilesize of (11754x11754). Is there any

> driver-independent way for getting this information?

 

You can use the opj_dump utility from openjpeg, or a GDAL Python script

https://svn.osgeo.org/gdal/trunk/gdal/swig/python/samples/dump_jp2.py

 

$ python swig/python/samples/dump_jp2.py /vsicurl/http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m/1/N5144H.jp2 | grep "siz\""

 

<Field name="Rsiz" type="uint16" description="Unrestricted profile">0</Field>

<Field name="Xsiz" type="uint32">11754</Field> <-- image width

<Field name="Ysiz" type="uint32">11754</Field> <-- image height

<Field name="XOsiz" type="uint32">0</Field>

<Field name="YOsiz" type="uint32">0</Field>

<Field name="XTsiz" type="uint32">11754</Field> <-- tile width

<Field name="YTsiz" type="uint32">11754</Field> <-- tile height

<Field name="Csiz" type="uint16">3</Field>

 

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tilesize in JPEG2000 and gdalinfo

jratike80
In reply to this post by jratike80

Thanks, Even.

 

This information may be less interesting once we have a released OpenJPEG driver that can handle big tiles properly.

The reason for my question was this error message that I got from my colleague who is cutting excerpts from a big VRT

 

Input file size is 1332094, 2304162

0...10...20...30...ERROR 1: Cannot decode tile, memory error

 

ERROR 1: opj_decode() failed

ERROR 1:

/vsicurl/http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m/1/N5144H.jp2,

band 1: IReadBlock failed at X offset 0, Y offset 0: opj_decode() failed

 

My guess is that error is due to this single tiled image.  Most JPEG2000 images in the VRT are written with tilesize 1024x1024.

 

-Jukka-

 

 

Lähettäjä: Even Rouault [mailto:[hidden email]]
Lähetetty: 5. syyskuuta 2017 12:09
Vastaanottaja: [hidden email]
Kopio: Rahkonen Jukka (MML) <[hidden email]>
Aihe: Re: [gdal-dev] Tilesize in JPEG2000 and gdalinfo

 

Hi Jukka,

 

>

> If I run gdalinfo against the JPEG2000 image in

> http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m

> /1/N5144H.jp2 the JP2ECW driver reports the blocksize as (256x256) and

> JPEG2000OpenJPEG driver reports that the block size is (1024x1024).

 

Yes for single tiled images, all JPEG2000 drivers will report a block size

of much smaller dimension to avoid GDAL consuming too much memory.

 

>

> What interests me would be to know the fact that this image has only one

> JPEG2000 tile with a tilesize of (11754x11754). Is there any

> driver-independent way for getting this information?

 

You can use the opj_dump utility from openjpeg, or a GDAL Python script

https://svn.osgeo.org/gdal/trunk/gdal/swig/python/samples/dump_jp2.py

 

$ python swig/python/samples/dump_jp2.py /vsicurl/http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m/1/N5144H.jp2 | grep "siz\""

 

<Field name="Rsiz" type="uint16" description="Unrestricted profile">0</Field>

<Field name="Xsiz" type="uint32">11754</Field> <-- image width

<Field name="Ysiz" type="uint32">11754</Field> <-- image height

<Field name="XOsiz" type="uint32">0</Field>

<Field name="YOsiz" type="uint32">0</Field>

<Field name="XTsiz" type="uint32">11754</Field> <-- tile width

<Field name="YTsiz" type="uint32">11754</Field> <-- tile height

<Field name="Csiz" type="uint16">3</Field>

 

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Tilesize in JPEG2000 and gdalinfo

Even Rouault-2

On mardi 5 septembre 2017 09:20:14 CEST Rahkonen Jukka (MML) wrote:

> This information may be less interesting once we have a released OpenJPEG

> driver that can handle big tiles properly.

 

Agreed. Sub-tile decoding with much more efficient RAM use is now implemented per

https://github.com/uclouvain/openjpeg/pull/1010 . It is currently under review

and should hopefully be merged into openjpeg master soon. A openjpeg 2.2.1 release

with it should follow. I'll keep the list informed.

 

In your use case, I see the image has RPCL progression order, which is, in theory,

not so bad for efficient sub-window decoding when using network access (PCRL would

probably be better here if doing only partial decoding at full resolution. RPCL allows

for efficient sub-resolution decoding), but currently openjpeg ingests the whole

codestream for the tiles it must decode totally or partially, so I/O wise,

in a network context, this is not friendly for big single tiled images. I know some

people would be interested in improving this. We'll see.

 

With openjpeg + above mentionned pull request 1010, I tested

 

gdal_translate /vsicurl/http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m/1/N5144H.jp2 out.tif -srcwin 0 0 1024 1024

 

Took 1m42 in total, for just 4s of CPU time. So mostly the download time of 150 MB

 

If you remove the /vsicurl/ prefix, then the image is completely downloaded in a single HTTP

GET request (through the intermediate HTTP driver), which enables generally better throughput

if you know that at the end the file will be entirely downloaded (so only for single

tiled images). Note that this will consume 2 times the file size of RAM (once by the HTTP driver,

and once by openjpeg)

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev